Cатсн²² (in)sесuяitу / ChrisJohnRiley

Cloud-based log Analysis and Visualization Raffael Marty

IaaS –> Infrastructure as a service

PaaS –> Platform as a service

SaaS –> Software as a service

LaaS –> Logging as a service!

It’s not that different from what we had before… Cost and time are the big saving here. No need to set up a huge datacenter just for your needs. No need to think about power, AC and what happens if you only need those 100 systems for a week!

Challenges

• Visability
• Big Data

Visibility

How many machines do you need to have running, what’s really being used.

Monitoring for performance, availability.

Security: New threats, new vulnerabilities and different risk distribution.

How can you effectively monitor a system if it comes up and goes down based on load? Whats happening on those systems?

IaaS –> Same as before

PaaS –> Lack of Infrastructure

SaaS –> Blind?

Big Data

How can you handle the large amounts of data created, including the logs of systems that might not exist long.

• NoSQL
• Distributed datastores
• Distributed queues
• Map reduce
• ETL (Extract, Transform, Load)
• …

Logging as a service!

Information Visualization

Better tools and capabilities are needed

There are a number of projects, but things still need to improve

A picture is worth a thousand log lines

What can visualization help with ?

• Exploration and Discovery
• Answer a question
• Pose a new question
• Increase efficiency
• Communicate information
• Support decisions
• Inspire (figure out how things work)

Important visualization…. monitoring webcam to tell you how full the team coffee pot is!

Visualization tools

2 categories:

• Reporting Libraries
  • HighCharts
  • Flot
  • Google Chart API (Sends data to Google)
  • Open Flash Cart
  • HTML5
• Visualization Libraries
  • TheJIT
  • Graphael
  • Protovis
  • ProcessingJS
  • Flare

Nowadays you can do almost everything you can do in Flash, with JavaScript!

A large number of these tools are now web-based.

<run through of some of the more useful tools>

The future is outsourcing the logging and concentration on how to output the stuff we need and visualize it in useful ways.

Old Skewl

Tailing a log, or multiple logs…

New Skewl

Centrally logging data, and using dynamic files to view visualizations of that data

Lack of data is also a trigger… why is there no output where there should be.

Changes over time can be easily seen using different colour intensity to show new or unseen traffic.

List: secviz.org/mailinglist

Twitter: @secviz