Discover more from Ankur’s Newsletter
Facebook's RoBERTa, the Next Best Advance in NLP
Here are five important AI stories from the week.
Since Google launched BERT late last year, there have been several improvements along the way such as OpenAI’s GPT-2 and XLNet. Facebook has launched its own improvement, RoBERTa, which produced state-of-the-art results on the most popular NLP benchmark known as GLUE. To build this, Facebook made some adjustments to Google’s BERT architecture but also trained on more data and for longer. Google’s and Facebook’s commitment to BERT matters because BERT is inherently a much more scalable solution, relying on semi-supervised NLP approaches (e.g., partially labeled datasets) versus the more mainstream supervised approaches (which requires lots and lots of hard to come by LABELED data).
To accelerate the field of autonomous driving and reinforcement learning and to integrate programmers more closely with its SageMaker offering, Amazon just launched a $399 miniature racing car. Amazon also launched a racing league for experts and hobbyists to compete.
Google’s Patrick Riley calls for greater scientific rigor in machine learning model development and productionization. Many times, machine learning works “great” in the lab and then does awfully in production. Common problems include splitting the training and test sets inappropriately, not explicitly modeling hidden variables (e.g., a seasonality component), and targeting the wrong objective (i.e., the model is developed to answer the wrong problem).
For novices and experts alike, Google has good resources on machine learning.
Machine learning is being successfully applied to many data-intensive fields such as autonomous vehicles, image and voice recognition, finance, marketing, and healthcare. Weather is a similar field with similar types of problems - there is too much data and that data needs to be analyzed and inferred from in near real-time to support forecasting. This article does a great job exploring how machine learning could help in forecasting weather better.
As machines become increasingly ubiquitous both in the household (Siri, Alexa, Nest, etc.) and in the workplace (via machine learning models and software), humans are becoming more trusting of automated decision-making by machines. In other words, performance of these machines is becoming more important than interpretability; even if you don’t know how or why a machine has decided a particular action, you come to accept it. Over time, this could lead to machines in the driver's seat, while humans just tag along for the ride. The New Yorker does a beautiful job discussing this in the article.
More Stories Worth Reading and Watching…