Nowadays, with the thrive of AutoML techniques, it is expected that Machine Learning algorithms will work well without any human intervention. This is why the biggest focus is on introducing new neural network architectures, especially those capable of learning to learn. Recently, the Data-Centric AI Challenge was proposed by Andrew Ng whose goal was to change the paradigm and instead of having a fixed dataset and modifying the model, now the model is fixed and the data is preprocessed so that the model results in the best performance. In my thesis, I would like to focus on another approach, where I would not modify the given data nor introduce new architectures, instead, I would like to propose new ways of injecting additional information into knowledge transfer models to increase their performance.