Facebook AI Research is open sourcing some of the conversational AI tech it uses to run its portal video chat display and M suggestions on Facebook Messenger.
The company announced today that PyTorch-based PyText NLP framework is now available to developers.
Natural language processing deals with how systems analyze people's languages and are able to make decisions and gain insight. The PyText framework, which the company sees as a channel for researchers to move faster between experimentation and deployment, will be especially useful for tasks like document classification, sequencing, semantic parsing, and multitask modeling, among others, "says Facebook.
The company has built the framework to fit quite seamlessly into research and production workflows with emphasis on robustness and low latency to meet the company's real-time NLP needs. The product is responsible for models that run more than a billion daily predictions on Facebook.
Another major highlight is the modularity of the frame, so it can not only create new pipelines from the bottom but to change existing workflows. PyText links to the ONNX and Caffe2 frames. It also supports training of multiple models at the same time as well as distributed training to train models across multiple races.
The company is obviously not done to improve its NLP framework. Facebook says it is still particularly important to work to build an end-to-end workflow for models running on mobile devices.
PyText is available at GitHub.