Digests » 144
this week's favorite
The black hat hacker’s premise is a story as old as time: find things, break things, make money. Over the years, there have been many creative attacks that have pioneered the industry and made groundbreaking changes in our digital infrastructure.
We explore the benefits of decreasing the input length of transformers. First, we show that initially training the model on short subsequences, before moving on to longer ones, both reduces overall training time and, surprisingly, gives a large improvement in perplexity. We then show how to improve the efficiency of recurrence methods in transformers, which let models condition on previously processed tokens (when generating sequences that are larger than the maximal length that the transformer can handle at once).
A helpful 4-page data science cheatsheet to assist with exam reviews, interview prep, and anything in-between. This resource is not meant to be a comprehensive deep dive into any specific model, but rather a quick refresher on a few of the most fundamental machine learning algorithms. The reader should have at least a basic understanding of statistics and linear algebra, though beginners may find this cheatsheet helpful as well.
In video games, nonplayer characters, bots, and other game agents help bring a digital world and its story to life. They can help make the mission of saving humanity feel urgent, transform every turn of a corner into a gamer’s potential demise, and intensify the rush of driving behind the wheel of a super-fast race car. These agents are meticulously designed and preprogrammed to contribute to an immersive player experience.
It is not hard to understand why people are receptive to a vision of the future in which AI’s primary impact is to augment human activity. At an elemental level, this vision leaves us humans in control, unchallenged at the top of the cognitive food chain. It requires no deep, uncomfortable reconceptualizations from us about our place in the world. AI is, according to this line of thinking, just one more tool we have cleverly created to make our lives easier, like the wheel or the internal combustion engine.