Static web pages - HTML stuffed with juicy user data - belong to the past and web 1.0. JavaScript frameworks like Facebook's React has drastically changed the digital landscape OSINT practitioners meet today. Nowadays HTML is just a mere shell - a blueprint - which remains to be populated with data, fetched when needed, i.e. when a user scrolls, clicks or navigates around. A consequence of this - and probably also legislation like the GDPR - is that still less user data is actually present when we inspect the HTML source of a given page. Finding user ids, timestamps and other necessary pieces of information becomes still more difficult with the traditional, old-school methods. The data simply seems to be gone.
This, however, is just an illusion. The data is still there. It's simply just invisible. The same structured, raw JSON data that these frameworks fetch from their servers and use to build the page on scrolls are increasingly not being stored in plain sight in the HTML. Instead it's stored as properties on the HTML nodes themselves. This talk aims to open the doors to the OSINT method necessary to extract large amounts of raw structured data from social media platforms exploiting the same techniques that giants such as Facebook, Instagram and TikTok actually uses themselves to access this data - but also hide it from the users. Using TikTok as an example the presentation will demonstrate how to locate and extract invisible data using JavaScript. How do you find the right nodes, that contain the invisible digital gold? How do you dump the data? On TikTok, on Facebook, on Instagram, on Twitter? This talk will teach you the basics you need to know to start your journey into the new reality of modern web development. Step by step we will explore a TikTok profile, dig through the HTML nodes and excavate the huge amount of awesome raw JSON data that TikTok stores invisibly behind the scenes. We'll write the few lines of JavaScript required to empty this amazing digital gold mine. Step out of the past and enter the future.
SANS Open-Source Intelligence Summit 2024
Uncover the Invisible Gold Mines: How to Dump Raw Data From TikTok
Jan Lauridtsen, OSINT Investigator, SpecialCrimes Unit, Danish National Police
View upcoming Summits: www.sans.org/u/DuS
29 авг 2024
