As online communication grows, memes have continued to evolve and circulate as succinct multimodal forms of communication. However, computational approaches applied to meme-related tasks lack the same depth and contextual sensitivity of non-computational approaches and struggle to interpret intra-modal dynamics and referentiality. This research proposes to a ‘meme genealogy’ of key features and relationships between memes to inform a knowledge base constructed from meme-specific online sources and embed connotative meaning or contextual information in memes. The proposed methods provide a basis to train contextually sensitive computational models for analysing memes and applications in semi-automated meme annotation.