Video platform and chatbot for historical education museum

Overview: The Client was in need of our expertise to develop an interactive online platform for historical education centred around the survival stories of prisoners in the concentration camps during the Communist era in Bulgaria. The general idea was to allow the users to have a virtual video conversation with the survivors by asking questions and receiving the

best answer from a pre-recorded set of answers to the most popular questions. They had drawn inspiration from a similar project about stories from Nazi-Germany concentration camps, but they didn't have the technical know-how on how to achieve it. A similar platform can be seen here. We worked in cooperation with multiple teams engaged with filming the interviews, video processing and graphic design.
 

Solution:

Phase 1 - Planning

We started with thorough planning of the questions for the interviews and the recording process since the team had only a couple of days to spend with each person and there was no room for mistakes. Every person had to be interviewed according to a prepared script and asked a very specific list of more than 500 questions such as "What is your name?", "What is your favourite colour?", "What do you remember about World War II?" and others.

Phase 2 - Data preparation

The video post-processing team took the challenging task to cut and edit the raw videos into a set of several thousand video answers. Our job was to categorize all of the video data by a weighted keywords approach and implement an algorithm that would match the keywords in the user question with the categorized answers in the database.

Phase 3: Algorithm development

Taking into account the grammar specifics for the English and Bulgarian languages, we made several configuration iterations to adjust the weights of the keywords and improve the performance of the algorithm. A team of internal and external testers helped to verify the results by trying out a large number of questions with different wording variations. All of the tests were monitored and actively used to constantly improve the algorithm. 

Once the platform recorded enough real-user data, the plan was to introduce an AI language model to allow for even more realistic and insightful conversations with the survivors. We would still use the keyword-matching algorithm to compare and measure the AI model performance, which would need significantly more data than we initially had at the start of the project. 

Phase 4 - Website development

When the algorithm was ready we continued with integrating it into the rest of the platform where users can log in, keep track of previous chat sessions and find out interesting information and facts about that particular historical period for further evaluation.
 

Future plans: Once the platform records enough real-user data, an advanced AI language model will be introduced to allow for even more realistic and insightful conversations with the survivors. The weighted keyword-matching algorithm is a good benchmark to compare and measure the AI model performance, which would need significantly more data than we initially had at the start of the project.