How does the DataSwarm system find the Zeitgeist?
The video below shows how the system works, graphically. Here it is tracking the key US election memes, and if you watch the graph dynamically you can see why we use the term “DataSwarm”
To summarise the video, the key points are:
The DataSwarm Analytic Engine was originally designed for the BBC who needed to “understand the Zeitgeist” in real time across all media output in the UK. The Zeitgeist is a grouping of a number of memes, and over time various memes emerge, gain or lose influence, or disappear. The system is designed to find the differences in memes and themes for any (or all) given topics – and since then our clients have been using it to gather insights for a variety of purposes. It uses Memetic Analysis to search for the Zeitgeist(s). The Zeitgeist is a collection of related memes, or ideas, that are interconnected.
There are some benefits to this approach vs existing Boolean approaches
- System gets to the key memes far faster, with far less work for the user – you don’t have to guess the issues and memes, the system finds them
- Sees memes systemically, ie understands their relationship to each other
- By tracking their movement in groups one can predict future trends more accurately
The graph shown in the video is also shown below, and works on 2 main axes:
–Relevance (Y axis) –The higher the relevance, the higher the connection between that meme and the Zeitgeist. For processing effectiveness, our system filters out less relevant memes. This level can be adjusted.
–Influence (X axis) – This shows how far a meme is travelling across the social graph, and how many people it can influence.
Each meme is a circle on the graph. The larger the circle the stronger the level of transmisson of the meme in the population considered.
Today’s main Zeitgeists are found in the top right of the graph. Over time you see a gradual progression of memes from the “Darwinian Stew” of ideas trying to gain traction on the left Y axis to the right, and as they gain adherence and “break out” they start to form tomorrow’s Zeitgeist. You also see a falling off of memes that once had traction. On the bottom (X) axis we track memes that are secondary, in that they are associated with the Zeitgeist being tracked but are not directly relevant to the topic. These are often a good way to find “sub-tribes”.
Client Applications to date include:
- Television/Media – Initially, the BBC needed “to understand the Zeitgeist” in real time across all their TV media output in the UK, and compared to other media operators.
- Customer Service and Satisfaction – For a large UK telecoms provider we combined social data memes with their customer services data to understand what elements of service mattered most, so its service operations could be optimised.
- Brand Marketing and Sales – A global cosmetics company asked that we identify Zeitgeists to understand the differences and similarities in future trends between markets across a range of products. For a global drinks supplier we located customer sub-groups via their secondary memes – for example one sub group listened to a certain type of music – this helped them to understand the sub cultures based on memetic insights to help develop campaigns with context.
Other Applications
- Competitor tracking and analysis, and comparison analysis is easy using memetic analysis
- The Engine can monitor the data swarms in real time – so it’s good for event warnings and crisis management
- It is fast so it can be used for large scale systems – so integrating and monitoring data sources in Smart City and Internet of Things applications is possible.
- In-company analysis of memes and tropes allows…
The Engine’s dynamic learning capability can be used to improve analytic outcomes over time, and it’s flexible – it can be adapted to create bespoke client solutions to help enterprises achieve their strategic goals.