The Bro Blog

Friday, October 10, 2014

Bro Monthly #2

Bro Monthly #2

Bro Monthly #2

Welcome to the 2nd Bro Monthly newsletter.
This month we cover the followoing topics:

  • Bro won a Bossie,
  • Bro.org needs help,
  • the Shellshock incident,
  • new features in the Intel framework,
  • news on BinPAC++,
  • Bro in research,
  • Bro in the wild,
  • Bro on demand.

Bro.org Needs Help

Bro has changed -- and improved -- a lot during the last years. Bro.org needs to keep pace with our developers and engineers, so we are looking for a web developer who can help us to give bro.org a facelift.
Please find all details on our jobs site.

Current Developments

Shellshock

The topic of the month was for sure the shellshock.
On September 24th the news went viral about a Bash patch that revealed a very bad vulnerability in Bash:
''...the vulnerability arises from the fact that you can create environment variables with specially-crafted values before calling the Bash shell. These variables can contain code, which gets executed as soon as the shell is invoked. The name of these crafted variables does not matter, only their contents.'' [from]
In other words, this bug allows anyone to execute their own code on affected remote hosts, in some cases even as root.

A Bro shellshock detector was released September 25th by Broala.

If this is all news to you, please stop reading here!
Patch your system NOW and use the Bro detector to see if you were attacked.
You are welcome to continue reading afterwards.

BinPAC++ Code Release

BinPAC++ is a next-generation parser generator that makes it easy to build parsers for network protocols, file formats, and more.
BinPAC++ is more than just a "yacc for protocols": it's an all-in-one system that enables developers to write attributed grammars defining both syntax and semantics of an input format inside a single comprehensive scripting language.

The BinPAC++ toolchain, built on top of HILTI, turns such grammars into efficient parsing code that exposes an well-defined C interface to its host application for feeding in input and retrieving results. At runtime, parsing proceeds fully incrementally—and potentially in parallel—on input streams of arbitrary size. Compilation takes place either statically at build time, or or just-in-time at startup.

You might have seen the name BinPac++ in the last Bro Monthly or even seen Robin Sommer's demo at BroCon'14. If not watch the video of the demo, get excited, and fetch the code from here.

Don't get too excited, though, because this is all still in prototype state, and not production-ready yet.

Intel Framework - New Features

Seth Hall published two new features for the Intel Framework this month:

  • The ability to extend the Intel log by handling the new Intel::extend_match event.
  • The ability to whitelist items with a new intel item field named "whitelist". If you want to start whitelisting intel items at runtime, you should create a new intel file with an extra "meta.whitelist" field and set the field value to "T" (there is a test that shows this). As you add elements to this intel file, those items won't show up in your log file.

Note that the script renames the intel.log to intel-ext.log.

Bro Research

Summary Statistics Framework

Last month, we presented our research paper on the Bro Summary Statistics Framework at the International Symposium of Research in Attacks, Intrusions and Defenses (RAID) in Gothenburg, Sweden. The Summary Statistics Framework allows the easy calculation of a wide array of statistics in real-time, independent for the underlying data. It is, e.g., used in Bro to detect port scans and brute-force attacks. It has a wide array of applications, like finding top traffic sources in your network, getting lists of the top DNS requests, etc.
For more details please Read the paper.

Heartbleed Study

Another research paper that was recently released is a measurement study about the Heartbleed SSL bug.
Among other measurement methods, the paper uses Bro to examine pre- and post-exploit network traces of several research institutions. The study used the Bro SSL analyzer to detect Heartbleed traffic in those traces.
For more details please refer to the publication.

Bro On Demand

Your call for better/more documentation is heard.
This month we improved the script language reference
and the documentation on the default logs.
We are constantly working on further documentation improvements.
Please use the community channels to let us know what is still missing in the
documentation.

Bro In The Wild

BruCon 0x06 presents their findings reviewing their network
using Bro. For everyone who always wanted to know what weird.log is good for, we recommend this blog post.
They find a lot by analyzing weird.log.

NodeJS has started a nice project called nodejs that can help you to get more out of your Bro logs.
''The idea is to do processing events from BRO IDS in nodejs - this is a simple first step by parsing the bro log files
'online' and generate new events when any of the logs gets modified.''

Bro Plugins from the outside
Anthony Kasza (OpenDNS) wrote another Brolog entry, this time about his first experience porting a script to a plugin. We are always working on our documentation, but being a part of the Bro team sometimes conflicts with writing down the ''right'' things to help others using Bro.
If you have trouble getting started with Bro plugins, Anthony's approach from the outside might help you.

Note that plugins require the current development version of Bro.
There's also some initial documentation on our web site.

Bro Teaching Community

The Bro Teaching Meeting is moved from every Tuesday to every second Friday 10 AM PST, starting 10/10.
If you want to join the Teaching community to learn more about teaching (with) Bro and share your experiences, write to info@bro.org.
The next Teaching Meeting will then be at 11/07 due to vacation.



Tuesday, September 16, 2014

Bro Monthly #1

Bro Monthly

Welcome to the 1st Bro Monthly, our new monthly newsletter covering the latest
developments in the Bro universe.
This newsletter will appear every month, around the 15th, as a Bro blog post.
Please send feedback, wishes, and suggestions to info@bro.org or @Bro_IDS on Twitter.


Events

BroCon'14


BroCon'14  was held at NCSA from August 18th - 20th. 
This year we received almost 150 attendees, our largest Bro event ever!
At this point we want thank again our sponsors:
Arista, Northrop Grumman, NSF, Reservoir Labs, and Security Onion Solutions.
A big thank you goes to NCSA who helped organizing the event.

We had great talks, presentations, and demos:
  • BroCon was opened by Adam Slagell, introducing the Bro Center of Expertise , an NSF project that enables a lot of new developments in the Bro universe,
    such as Bro Live! and Try.Bro (see below).
  • Nick Buraglio from ESnet talked about "Best practices for securing the science DMZ".
  • Bob Rotsted from Reservoir Labs discussed the "Value of context when detecting adversaries".
  • Johanna Amann from ICSI presented the new SSL analyzer in Bro 2.3 that is also capable of detecting the Heartbleed exploit.
  • Michael Pananen from Vigilant Technology Solutions showed how he automated Bro's installation, upgrade, and configuration using puppet.
  • Kurt Grutzmacher from Cisco Security Solutions presented OpenSOC, a Hadoop solution to extend Bro's ingestion capacity to 1.2 million packets per second and more.
  • Aashish Sharma gave some very entertaining insights into his day-to-day work fighting off attacks at LBNL.
  • Matthias Vallentin from ICSI introduced VAST (Visibility Across Space and Time), a large-scale network forensics platform.
  • Robin Sommer's (ICSI/LBNL/Broala) live demonstration of the new BinPAC++ parser generator was one of the most resonating contributions. He implemented a full protocol parser in less than half an hour in front of the audience.
  • To conclude the day Seth Hall (ICSI/LBNL/Broala) talked about the future of Bro, giving insights into long term and short term plans.
  • The third day was opened by Bob Bregant from the University of Illinois, who talked about how Arista's "DANZ" software can be used in combination with Bro to balance the costs when monitoring large high speed networks, working around problems arising from aggregation and traffic splitting.
  • The third day was wrapped up by a panel discussion in which the audience had the chance to pick the Bro team's brains about their visions for the Bro project.
Apart from the talks and demos we had five exercises ranging from beginner level to quite advanced scripting challenges.
The exercises can be found at the event site of BroCon'14 .
The solutions will be given out on demand. Please contact info@bro.org.

The videos of most of the BroCon'14  talks are now online.
The Bro team respects the privacy preferences of our speakers, so when a speaker opted to not being recorded, we do not offer a video of the talk.


2014 NSF Cybersecurity Summit for Large Facilities and Cyberinfrastructure


The CTSC Summit was held in Arlington, VA on August 26th - 28th.
The Bro team presented a one-day training to a smaller group of attendees affiliated with NSF projects.  The training consisted of a couple exercises from BroCon '14 as well as some presentations.
Robin Sommer also gave an overview of the Bro Center of Expertise at the main event on August 27, 2014, in which he presented our latest efforts for making Bro more accessible to the community, and enable people and institutions to use Bro more effectively.

Bro Commits

Bro 2.3.1


Bro v2.3.1 has been released. This release addresses a potential DOS vector using specially crafted DNS packets.
It also fixes a bug in the OCSP validation code that could lead to crashes as well as a memory leak.
The source distribution and binary packages are available on our downloads page.
See CHANGES for the full commit list.
Since this release addresses a bug fix, we encourage users to review and install at their earliest convenience.
Feedback is encouraged and should be sent to the Bro mailing list .

Bro's new dynamic plugin infrastructure


Any who has tried to add a new protocol analyzer to Bro will havenoticed that so far that has required touching a lot of pieces of Bro, as well as a complete rebuild of the Bro code base. We have just added a new comprehensive plugin infrastructure to Bro that makes this process much easier by allowing to write protocol analyzers externally, without *any* changes to the Bro core, by compiling them into a shared library that Bro will then load at runtime.  That way, the custom code remains self-contained, and can be maintained and installed independently.

This new infrastructure is in fact not limited to protocol analyzers, but supports other components of Bro as well. Developers can now use plugins also to provide custom file analyzers, log writers, input readers, packet sources and dumpers, as well as new built-in functions. For more information, see the introduction to writing Bro plugins.

Packet Bricks


We are happy to present an initial prototype of Packet Bricks, a new Bro-related project written by Asim Jamshed from KAIST, who visited the Bro team in Berkeley over the summer.

Packet Bricks - which is still under active development - is a Linux/FreeBSD daemon that is capable of receiving and distributing ingress traffic to userland applications. Its main responsibilities will eventually include (i) load-balancing, (ii) duplicating and/or (iii) filtering ingress traffic across all registered applications. The distribution is flow-aware (i.e., packets of one connection will always end up in the same application). Packet Bricks leverages the netmap packet I/O framework for handling packets efficiently, and employs netmap pipes to forward packets to userland applications.

Packet Bricks is available on github. It's still a very early piece of software, and we announce it at this time primarily for user's willing to help us collect some first experiences with it. If you have any feedback, please send it to the Bro development mailing list. If you aren't subscribed yet, you can
do so here.

Bro On Demand




Bro Teaching and Trying

In August we launched three new projects that aim at helping the Bro community use, learn, and teach (with) Bro.

The Bro Teaching Community


We are happy to announce the newly started Bro Teaching Community, a community project of educators interested in collaboratively exploring Bro's use as a teaching tool, and sharing experiences and material. The goal is to create a knowledge base and resource collection for educators,
ranging from example curricula and slide sets to exercises for all purposes and skills levels.
We provide logistics and technical advice, e.g., weekly calls, a mailing list, a repository with seed material, and access to the Bro team.
To learn more please visit our Teaching Site.

The Bro Playground


The Bro Playground  is a new part of the Bro Community resources.
It is a collection of tools and toys to assist you.
Whether you want to teach Bro, use Bro for teaching others, teach yourself, or try something out “quickly” without impacting your live system, this is the place to look for the right tool for your use case.

Try.Bro


Try.Bro - as simple as that!
Try.bro is a web-based Bro scripting sandbox made freely available to users on our site

No login.
No installation.
No trouble.

We have included a few basic scripts and pcaps to help get you started.
You can paste your own scripts or upload your own pcaps, too.
We even included the option of chosing different Bro versions to test your scripts against current or previous releases.
And, last but not least, Try.bro temporarily caches your work and generates a
unique URL to share with others.
No more copying and pasting scripts or log files, just send the link.
We store code fragments for three days and pcaps for one hour.
The timeout is reset when the link is used.

To learn more please refer to the blog post.

Bro Live!


We are excited to announce the public release of Bro Live!

Bro Live! is a training system that gives users hands-on access to a Bro
learning environment without having to download a virtual machine or its
required dependencies.  Bro Live! may be built with exercises for a given
class or workshop and access to the environment may be limited to the
duration of the event. All the user needs is an SSH client with access to the Internet.

Please read our latest Blog post.

Cool Stuff

Exfil Framework by Reservoir Labs


Robert Rotsted from Reservoir Labs posted on the Bro mailing list about the new Exfil Framework.

"The Exfil Framework is a suite of Bro scripts that detect file
uploads in TCP connections. The Exfil Framework can detect file uploads in
most TCP sessions including sessions that have encrypted payloads (SCP,SFTP,HTTPS).
The scripts are located at:







Monday, September 15, 2014

Announcing Bro Live!

We are excited to announce the public release of Bro Live!

Bro Live! is a training system that gives users hands-on access to a Bro learning environment without having to download a virtual machine or its required dependencies.  Bro Live! may be built with exercises for a given class or workshop and access to the environment may be limited to the duration of the event. All the user needs is an SSH client with access to the Internet.

The inspiration for Bro Live! came from the frequent frustrations we faced when building and distributing virtual machines for our training events: files are too large, the variety of user platforms and virtualization products complicate the troubleshooting process, pushing last minute updates are difficult, too much time spent not learning Bro, etc.  Bro Live! overcomes all of these limitations by putting the responsibility back in the hands of the system engineers. Users simply open a terminal, connect to the server via ssh, create an account, and they are up and running.

How Does Bro Live! work? Bro Live! was built using Linux, OpenSSH, Docker, and Bash to glue everything together. It relies on Linux-based containers, managed by Docker, to place users in their own isolated environments with shell access to Bro, the exercises, and the standard Unix toolset. The user's work is saved in their container typically for the duration of the training event and can be easily re-attached at anytime during the event to continue their work.

Want to host your own Bro training event with a system like this? See our work in Github for more information.  As always we welcome your feedback and bug reports, contact us at info@bro.org, follow us on Twitter, or like us on Facebook.

Thursday, September 11, 2014

Bro and Chrome's Sunsetting of SHA-1

A few days ago, Google announced their plans for sunsetting certificates using the SHA-1 hash algorithm in the near future. Google does not think SHA-1 certificates should be considered secure in the future anymore as collision attacks against SHA-1 get more realistic.

For these reasons, they will start reducing the security indicators if the certificate chain uses SHA-1 and the host-certificate is valid beyond the end of 2015. Depending on the exact validity in the certificate, the visual display will first change to a symbol showing slight errors in the HTTPS connection to symbols showing the connection to be not secure.

Chrome will start reducing the security indications for such certificates at the end of September 2014.

We created a Bro script that can detect servers in your local network that contain such certificates.

To use the script, simply download it from https://github.com/0xxon/bro-scripts/blob/master/chrome-sha1.bro and either load it in your local.bro (when using broctl) or add it to the Bro command line when starting it manually. By default the script only checks servers that are in your Site::local_nets list.

The script will output notices for each server containing such certificates to notice.log. An example output is included in the git repository.

Please let us know any issues you encounter when using this script.

Monday, September 8, 2014

Bro 2.3.1 Release

Bro v2.3.1 has been released.  This release addresses a potential DOS vector using specially crafted DNS packets.  It also fixes a bug in the OCSP validation code that could lead to crashes as well as a memory leak.  The source distribution and binary packages are available on our downloads page.

See CHANGES for the full commit list.

Since this release addresses a bug fix, we encourage users to review and install at their earliest convenience.

Feedback is encouraged and should be sent to the Bro mailing list.


The Bro Team

Friday, September 5, 2014

Announcing Try.bro

We are very excited to announce the official launch of Try.bro.org!



Try.bro is a web-based scripting sandbox made freely available to users on our site.  No login.  No installation.  No trouble.

We have included a few basic scripts and pcaps to help get you started.  You can paste your own scripts or upload your own pcaps, too.  We even included the option of version control to test your scripts against current or previous versions of Bro.  And, last but not least, Try.bro temporarily caches your work and generates a unique URL to share with others.  No more copying and pasting scripts or log files, just send the link.  We store code fragments for three days and pcaps for one hour.  The timeout is reset when the link is used.

How does Try.bro work?  Mainly we use Docker to run Bro in an isolated container.

In the few weeks we have been beta testing Try.bro it has become an extremely useful tool for collaboration, troubleshooting problems with users, and more.

We have additional features planned for Try.bro, including making it embeddable and building scripted tutorials around it for guided learning.

If you find a cool use for Try.bro or want to request a feature/report a bug, please let us know.

Contact us at info@bro.org, follow us on Twitter, or like us on Facebook.

Thursday, August 7, 2014

Meet the Bro Teaching Community

We are happy to announce the newly started Bro Teaching Community,
a community project of educators interested in collaboratively
exploring Bro's use as a teaching tool, and sharing experiences and material.
The goal is to create a knowledge base and
resource collection for educators, ranging from example curricula and
slide sets to exercises for all purposes and skills levels.

We invite you to participate in our open discussion every
Tuesday at 10:00 AM PDT. In these meetings we discuss planned curricula,
practical and technical topics around exercises, slide sets, and general
questions related to teaching security, networks and systems with Bro

For details see www.bro.org/teaching/ or contact us directly via info<at>bro.org