Oh, I can see this going crazy in all kinds of ways. It's going to be a scary future, but it's going to be fun!

Yeah, in case it finds a security vulnerability in the VM managing software (like Virtualbox or something that) and ends up going too far. It'd really need a real no-Internet-connection-at-all air gap between the Internet and the computer. But in general, it'd be interesting if Microsoft could just put a Windows box on a network and have another machine on the same network just keep trying to get in with whatever it can think of. If there's even a single security vulnerability, then it'll eventually find it, and the engineers can then figure out what happened patch it up. Given enough time, they could (hypothetically) weed out every security vulnerability.

This kind of thing could get really exciting! The only thing that's ever held machine learning and neural networking back from the rest of computer science is that we didn't have hardware that was powerful enough for it. Now our computers are starting to get powerful enough for it! Things are going to start getting very exciting in the next few years, mark my words!

Hadn't even considered that! You won't find any interesting bugs for a while. Leaving it running for a few months could show us some interesting stuff!

Now imagine pointing this kind of AI at regular software and trying to find security vulnerabilities! The more I read about it, the more I realize that neural networking and machine learning really could be the future!

Considering how smart it got after just a few hours of learning, you probably wouldn't even see a noticeable difference between a week of training and a year of training because a week of training would be so good. There was a similar research project that came out of Google DeepMind (I think) a few months that involved a similar idea, but with lots of different games for an old Atari console that I can't remember the name of. I downloaded it and left it running on one game for three hours and it became incredible at the game. I left it running some more for two whole days and it wasn't much better, mainly because there really wasn't any room for improvement after a few hours.

If the software in this video is as good as it looks, then there (probably) isn't much room for improvement after a week of training the model. That said, a year of training on a computer with a super fast processor would still be fun, just in the shits-and-giggles department.