What is OSINT?

OSINT is a phrase you’ll hear about in the cybersecurity community. It’s an essential skill and methodology for researchers and defensive security professionals. So what is it? What is it all about?

Open source intelligence

OSINT stands for open source intelligence. The “open source” part refers to publicly available information, and “intelligence” refers to finding relationships between individual pieces of information from which we can create specific patterns and profiles about the target. OSINT is a very broad area, and there are many different ways to do it.

Why do people do OSINT?

Here are some of the reasons why people engage in OSINT:

• To learn about employers or job applicants before hiring

• To learn about people before dating or making friends with them

• To make consumer or corporate purchasing decisions

• To engage in many different areas of academic work

• To learn about cybercriminal activity

• To learn about specific cyber threats

• To conduct old-fashioned criminal investigation for law enforcement

• To find missing people or lost family members

• To plan holidays or business trips

• To facilitate pentesting, to gather information on your pentesting target

And the list goes on and on.

OSINT sources

There are almost as many sources of OSINT as there are reasons for doing OSINT:

• Books, newspapers, magazines, and public documents in libraries

• Publicly available databases from government agencies and private organizations

• Archives of television, radio, and online news reporting

• Social media platforms such as Facebook and LinkedIn

• Genealogy databases

• All areas of the clearnet (the “normal” internet) that can be legally accessed

• All areas of the darknet (the parts of internet that require Tor or I2P use, including the Dark Web) that can be legally accessed

What isn’t OSINT?

OSINT is passive and lawful research. OSINT is based only on the passive gathering of information. So the moment you have to directly ask someone for information or initiate a scan that interacts with the target, that isn’t OSINT. That’s active research. And if your research requires breaking the law or otherwise accessing information you aren’t permitted to access, that isn’t OSINT either. Cyber attacks such as brute forcing and spyware usage, and any espionage conducted by civilians and without a police search warrant are both highly illegal pretty much everywhere in the world. That definitely isn’t OSINT.

How do I OSINT?

I really enjoy doing OSINT work. My professional experience with OSINT is my side gig doing cyber threat research for a major bank. Here’s what I’m allowed to publicly say about it. 

Cybercriminals often use the Dark Web to both discuss criminal activity and engage in it. They often plan cyber attacks with other people using Dark Web forums, and they sell malware, databases of stolen and sensitive information, and cyber attack services on both Dark Web forums and darknet markets. These types of sites function similarly to Reddit or eBay on the clearnet, but unlike Reddit and eBay, they’re used to break the law. The reason why the Dark Web is attractive for cybercriminals is because it’s much more difficult (but not impossible) to link criminal activity to individuals and groups through the darknet than it is to identify people on the “normal” internet. The various encrypting network nodes through the Tor and I2P networks anonymize users and obfuscate identifiers which can be used to identify computers, mobile devices, LANs, and ISPs, such as IP addresses and MAC addresses. Tor and I2P are used for a lot of lawful, ethical, and innocuous activity as well, but they also inadvertently facilitate illegal activity. 

What I do is I go on these Dark Web sites and look for information about cybercrime which specifically targets the bank I’m working for. Here in Canada, using the Dark Web is generally legal, as long as you don’t engage in activity which would also be illegal on the clearnet or offline. I take a lot of screenshots that help me write lengthy penetration testing reports on any evidence that I find, along with my analysis. I never directly interact with cybercriminals, I’m “lurking.” So yes, what I do is definitely OSINT!

OSINT tools

There are a wide variety of tools you can use for OSINT. This isn’t a complete list, but here are some of the more commonly used OSINT tools:

• Your local library

• Your web browser

• Tor and I2P clients

• Shodan is a special kind of search engine. Instead of searching websites, you can use Shodan to find publicly available information on devices and networks that are connected through the internet. This includes everything from home and office LANs, to endpoints like PCs, tablets, and phones, to Internet of Things devices, internet-connected cameras, microphones, traffic lights, internet servers, and the list goes on. A defensive security professional who is securing a sensitive endpoint, server, or network will do what they can to make sure they aren’t searchable by Shodan.

• Maltego is an OSINT tool that’s often used by cybersecurity researchers and law enforcement. The platform takes data from a variety of different online data sources (“over 58 data integrations from over 35 data partners”), from Have I Been Pwned to Blockchain.info. The graphs and other kinds of visualizations the application outputs from your search queries can be highly customized. Maltego is best for visually simplifying patterns from huge amounts of data.

• Recon-ng is a web reconnaissance tool which can be used to extract all kinds of specific data on web targets.

Get into OSINT with HTB Academy

If you want to learn how to do OSINT and get hired for jobs which require OSINT skills, HTB Academy is the best place to start. 

OSINT: Corporate Recon will teach you a universal approach, methodology, and what you need to know about OSINT for pentesting:

“OSINT (Open-source Intelligence) is a crucial stage of the penetration testing process. A thorough examination of publicly available information can increase the chances of finding a vulnerable system, gaining valid credentials through password spraying, or gaining a foothold via social engineering. There is a vast amount of publicly available information from which relevant information needs to be selected.”

While you’re here, check out my interview with the man who created the course, Senior Training Developer Valentin Dobrykov (Cry0l1t3).