In my thesis, I am exploring the effect of dialogic interaction in the effectiveness of human-AI creative collaboration. Dialogicness is defined as a feedback loop between humans and AI where there can be interaction through and about the artifact, that moves the artefact closer to completion. There are some conditions for effective dialogue. Initially, we developed a [[A typology of Human-AI Dialogic Creative Interaction]], where we outlined the importance for the AI collaborator to know about the human's goals and context. Over the course of my research, I have found the importance of the agent having context as a crucial component of effective interaction. As dialogic interaction has become the norm, largely thanks to chatGPT, many of the challenges of turn based feedback communication have been addressed successfully (though not all, see [[Interaction through and about, an important dialogic affordance]]). However, an important challenge that I have identified is AI collaborators that have context. See [[The main challenge for AI collaborators is lack of context and specific knowledge about particular use cases]]. For example, an AI design collaborator would want to have context in the form of: - Memory about previous interactions over the long term - A knowledge of the designers philosophy, vocabulary and aesthetic preferences - Knowledge about the field, and specific techniques used by the human designers - Overarching goals of the user, and specific ones at interaction time Another example could be an AI law collaborator that helps lawyers craft contracts. The AI agent would need context about: - The legislation in that particular area - Preferences of the lawyers - Context about specific cases Large language models have spawned a multitude of AI collaborators. However, generative language models out of the box hallucinate facts, and don't retain knowledge from previous conversations. Agents that can reference existing knowledge bases have become an increasing area of interest. In my research, I have found agents that can accurately reference existing knowledge bases and retain context might be more effective collaborators. I have found this in three cases: ## Data sonificaton AI collaborator First use case: Canberra installation 2021 [[System of a Sound - An interactive AudioVisual Installation using large Language Models, data sonification and pose recognition]] Use an LLM to convert data into text. Limitation, no context of the wider creative goal. Use LLM embeddings to produce matchings. Second use case: Opera House, 2022 Use LLM to convert data into text, but maintain wider context. Overall goals variability. Paper looks at how to enable this memory. ## Tilly First iteration Design collaborator. Used to co-design, explain pieces and gather feedback. Limitations: - No memory of interactions - No specific knowledge beyond the LLM knowledge and that included in prompt Second iteration Active conversation, memory, knowledge of the creative goals. Paper looks at: practice based case study, on how to enable this memory in LLMs. ## Enabling memory How is memory enabled? Vector databases. Challenges of vector databases: only chunks of text. Is there a better way of doing it? https://github.com/karpathy/randomfun/blob/master/knn_vs_svm.ipynb For complex information retrieval. Relevant docs