Meena is the Google’s new chat bot model

Researchers from Google Brain have presented Meena – the new state-of-the-art chatbot that can chat almost about anything.

In their new paper, named “Towards a Human-like Open-domain Chatbot”, researchers describe their novel neural network model with 2.6 billion parameters that outperforms all existing conversational models. The new model is based on the popular Transformer seq2seq architecture, and its specific architecture was discovered using evolutionary neural architecture search, with the single objective of improving perplexity in the conversations. The evolutionary search algorithm produced an architecture in which there was a single Transformer encoder block and even 13 Transformer decoder blocks. Researchers hypothesize that Meena achieves higher conversational quality due to its powerful decoder module.

The model was trained using conversation threads which were organized as tree threads and samples contained 7 context switches between participants. The data used to train Meena, was coming from social media conversations and in total it was 341 GBs of text, which was previously filtered.