[Request] Can anyone show me how to scrape videos?

1    26 Jul 2018 00:41 by u/Salbuchi_2019

A good guide will be enough. I just wan't to be able to scrape any video I bump into the Internet.

I know how to get to the div with the video URL but there's some kind of hot-linking protection in the CDN. I'm not sure how to proceed.

I'm interested in doing it from the browser's developer tools, not with external tools if possible (other than specific webdev browser extensions).

3 comments

0 u/SwiftLion 26 Jul 2018 01:08

I can help push you in the right direction, at least.

When you see that loading a content URL directly fails, it means there is some sort of access mechanism that goes beyond the URL itself. My guess is that it’s in the headers. It may be something like JWT, but even if it’s dynamically calculated, it’s probably valid for a little while.

Download Charles, HTTPScoop, or similar. Filter the sniffing down to the URL you know is the video. Then, compare the stuff you see in their request that works with what you see in your own that doesn’t. That’s at least gonna get ya started.

0 u/Salbuchi_2019 [OP] 26 Jul 2018 01:48

Great! I used Telerik Fiddler and managed to get the video. Thanks!

0 u/SwiftLion 26 Jul 2018 12:42

Glad to help!