Recently, I have written several reconnaissance-oriented posts. In this one, I want to follow the trend and present what I think might be one of the (first) guides for Censys.io.
I assume you are at least somehow familiar with Shodan. It allows you to search for public-facing hosts with a friendly interface and several filtering options. From my view, Shodan was optimized to provide search capabilities for IoT devices across the globe. Its UI includes features such as current view of an exposed camera, VNC screenshots, and RDP screenshots. I always thought of Censys as an "academic brother" of Shodan, with less functionality and worse UI. That could not be further from the truth.
Censys provides a more complicated interface and filtering options than Shodan. In general, this would be a downside, but I think that for cyber security practitioners it is a huge advantage. The central part of the website is a search box, which divides the search into three separate areas:
- IPv4 Hosts
- Websites
- Certificates
ElasticSearch powers its search functionality (at least their documentation for searches points to Elastic's official documentation). The searches follow classic boolean-based queries. In the following sections, I explain its key-value system and differences between the three search areas.
IPv4 Hosts
One of the primary purposes of Censys is to scan IPv4 space, find open services, and gather the provided banners.
As you can see, this is an example of IPv4 result page. The host has three services open: 22/ssh, 23/telnet, and 80/http. The cool thing about Censys is that it allows you to filter based on protocol-specific fields. For instance, you can filter based on HTTP response status code for 80/http and at the same time filter based on a public key for 22/ssh.
Search queries are build in the following way:
The query above represents hosts which returned status code 404 for GET "/".
Some fields are not attached to any particular protocol. Scan metadata are also included:
- Timestamp (updated_at.)
- Tags (tags.)
- Location data (location.)
- Autonomous System data (autonomous_system.)
- ...full listing of these keys in this table
I briefly mentioned tags. Tags are specific values attached to some hosts. Censys includes heuristics that try to recognize interesting hosts based on their banner response and other factors. Tags are then used to cluster hosts based on some specific factor. Tags include:
- "camera" — Public-exposed cameras
- "nas" — As for Network-Attached Storage
- "raspberry pi" — I think this is self-evident
- ...full listing here
You can indeed stack multiple key-value pairs together using "AND" or "OR". Let's look at several examples:
Hosts that have SSH and telnet open:
ports:22 AND ports:23
Hosts in Slovakia which are tagged as a camera:
location.country_code:"SK" AND tags:camera
OpenSSH servers running on Debian across Europe:
22.ssh.v2.metadata.product:"OpenSSH" AND metadata.os:"Debian" AND location.continent:"Europe"
Personally, I like to open JSON output (like this) for some host which shows all key names. The query can be built much faster if you see what you want to filter. Censys also provides a way to specify ranges:
Hosts that were updated from May onwards and returned 5xx HTTP code:
updated_at:[2018-05-01 TO *] AND 80.http.get.status_code:[500 TO 600]
Regular expressions and wildcards are also possible. Full syntax can be found here.
I also recommend switching to Metadata tab which reveals an overview of the result set:
Websites
Websites provide pretty much the same view as IPv4 hosts. Censys currently scans all domains in Alexa 1 million. Information about certificates and DNS (e.g., AXFR checks) are added on top of regular host scan.
Certificates are explained in the section below; here I want to mention my favorite filter based on DNS data. The key "0" is available in Websites view and enables to filter on DNS data.
All domains having external DNS zone transfer allowed:
0.lookup.axfr.support:true
The results were surprising because, in the result list, there were some well-known domain names. Nevertheless, they indeed allowed AXFR requests from random hosts. Weird. You can read about DNS zone transfer and its implications here.
The only limitation of Websites view seems to be the number of scanned domains. Hopefully, Censys can integrate results from Project Sonar soon.
Certificates
Initially, I thought that Censys certificate capability suffers from the same problem as I explained in Project Sonar. Server Name Indication (SNI) allows to host multiple certificates on a single machine and present only one based on domain name presented during SSL handshake. When scanning IPv4 space, the scanner does not provide the domain; it just provides IPv4 as a hostname. This yields to observing only a fraction of certificates present on the scanned hosts.
I was pretty surprised to see that only 20% of certificates in Censys comes from SSL scans on IPv4 address space. The rest comes directly from Certificate Transparency (CT) logs.
I do not want to go deep in explaining Certificate Transparency (CT). If you are not aware of it, you can read more here. CT project was created to gain visibility into what certificates are CAs issuing. What does it mean to us? The process looks as follows:
It means that as soon as the certificate is issued, the certificate (and more importantly its SubjectAltNames) are provided nearly in real-time to Censys. Censys thus doesn't need to rely on IPv4 scans to discover certificates.
Let's make a small detour and talk about similar services that offer visibility to certificates. First one is crt.sh. It is a well-known service in the community that provides search capability on top of CT logs. However, compared to Censys, interface in crt.sh allows only basic filtering options which are usually not sufficient when doing more complex analyses. As far as I know, crt.sh relies only on CT logs whereas Censys combines it with IPv4 scans as well. Similar services are also certspotter.com and certdb.com.
Next on the list is CertStream service which offers certificates in real-time. It has several advantages such as processing the data using event-driven programming and processing the received input. On the other side, it does not offer an option to search in history.
I think that Censys currently offers the best free service for certificate search.
Censys offers several new keys for certificate search:
- Certificate fields (parsed.)
- Certificate metadata (metadata.)
- Trust chain (validation.)
- Certificate Transparency (ct.)
- ...full listing here
Again, I recommend opening one sample certificate to see the most interesting key names yourself. Let's look at a couple of examples:
Certificates for www.example.com:
parsed.names:"www.example.com"
Certificates that are valid by Apple but not by Mozilla NSS:
validation.apple.valid:true AND validation.nss.valid:false
Certificates issued by Symantec that were seen during SSL scan:
parsed.issuer.organization.raw:"Symantec Corporation" AND metadata.seen_in_scan:true
Certificates issued by Let's Encrypt in 2018:
parsed.issuer.organization.raw:"Let's Encrypt" AND parsed.validity.start:[2018-01-01 TO *]
Censys also offers interesting pivot/discovery techniques. You can search based on several common factors:
I want to emphasize that I have not seen such functionality in any other (free) service.
Discovering SCADA
Now let's try to find something interesting. As my first mission, I chose to search for SCADA systems across the globe. Hopefully, you have a pretty good idea of how to structure your search query.
The easiest solution is to scan the IPv4 hosts based on several criteria. Luckily, we can make use of handy tags to filter based on:
tags:"scada"
We can filter further down based on manufacturer:
metadata.manufacturer:"Schneider Electric" AND tags:"scada"
Alternatively, there are similar tags such as JACE, fox, or modbus. You can play with different keys to discover the host of your choosing. As I said before, Shodan provides better functionality regarding IoT visualization, but Censys offers more filtering options.
Discovering Phishing Sites
I like phishing_catcher. The only downside is that it takes data from CertStream which is not so ideal if you are trying to find sites that might be a couple of weeks old.
As I mentioned in the sections above, Censys gathers a massive amount of certificates, so we can utilize its capability to find domains having (likely) malicious intents. Firstly, let's think about how phishing domains are usually created. Attackers want to convince the victim that the site she is visiting is indeed legitimate. Therefore, SSL certificate, and more importantly "Secure" flag in URL bar is a must. Since Let's Encrypt offers free certificates, it is very likely that the adversaries go this way. We have our first marker.
Secondly, the domain name. It will most likely contain the service which adversaries are attacking as a substring (e.g., "apple", "bankofamerica", "paypal", ...). I highly recommend checking out this wordlist to see which services are most likely targeted. Doppelganger and typosquatting domains are also used, however recently it seems that the trend is to have very long FQDN to deceive the victim about what is a second-level domain. The domain name is our second marker.
Let's now try our hypotheses:
(apple.com*) AND parsed.issuer.organization.raw:"Let's Encrypt" and parsed.validity.start: [2018-05-03 TO *]
We can see that there are some questionable domains in the results. Overall, 126 certificates were containing "apple.com" issued on 2018-05-03. Let's try visiting one of them:
Yikes. It seems that Safari is already aware of it at displays warning. After ignoring the warning, the web server was not responding. Some phishing sites might respond only to requests from a specific country or specific URL path.
We can continue and search for PayPal phishing sites:
(paypal.com*) AND parsed.issuer.organization.raw:"Let's Encrypt"
Hopefully, you now have a rough idea of what is going on here. I urge you to replace "paypal.com" with the domain of your organization. If you discover some phishing site, it is also useful to search for the same Public Key in the certificates database. Not-so-much-skilled adversaries sometimes tend to reuse key-pairs in multiple certificates.
Until next time!
