Cатсн²² (in)sесuяitу / ChrisJohnRiley

Because we're damned if we do, and we're damned if we don't!

DeepSEC: Cloud-based log Analysis and Visualization

Cloud-based log Analysis and Visualization Raffael Marty

IaaS –> Infrastructure as a service

PaaS –> Platform as a service

SaaS –> Software as a service

LaaS –> Logging as a service!

It’s not that different from what we had before… Cost and time are the big saving here. No need to set up a huge datacenter just for your needs. No need to think about power, AC and what happens if you only need those 100 systems for a week!


  • Visability
  • Big Data


How many machines do you need to have running, what’s really being used.

Monitoring for performance, availability.

Security: New threats, new vulnerabilities and different risk distribution.

How can you effectively monitor a system if it comes up and goes down based on load? Whats happening on those systems?

IaaS –> Same as before

PaaS –> Lack of Infrastructure

SaaS –> Blind?

Big Data

How can you handle the large amounts of data created, including the logs of systems that might not exist long.

  • NoSQL
  • Distributed datastores
  • Distributed queues
  • Map reduce
  • ETL (Extract, Transform, Load)

Logging as a service!

Information Visualization

Better tools and capabilities are needed

There are a number of projects, but things still need to improve

A picture is worth a thousand log lines

What can visualization help with ?

  • Exploration and Discovery
  • Answer a question
  • Pose a new question
  • Increase efficiency
  • Communicate information
  • Support decisions
  • Inspire (figure out how things work)

Important visualization…. monitoring webcam to tell you how full the team coffee pot is!

Visualization tools

2 categories:

  • Reporting Libraries
    • HighCharts
    • Flot
    • Google Chart API (Sends data to Google)
    • Open Flash Cart
    • HTML5
  • Visualization Libraries
    • TheJIT
    • Graphael
    • Protovis
    • ProcessingJS
    • Flare

Nowadays you can do almost everything you can do in Flash, with JavaScript!

A large number of these tools are now web-based.

<run through of some of the more useful tools>

The future is outsourcing the logging and concentration on how to output the stuff we need and visualize it in useful ways.

Old Skewl

Tailing a log, or multiple logs…

New Skewl

Centrally logging data, and using dynamic files to view visualizations of that data

Lack of data is also a trigger… why is there no output where there should be.

Changes over time can be easily seen using different colour intensity to show new or unseen traffic.



Comments are closed.

%d bloggers like this: