Intro#

These instructions are for logstash-1.1.5-monolithic.jar.

Often it is tricky to get LogStash running for prototyping reasons. The following instructions will get you going.

Note: These instructions are for getting LogStash to read from stdin and a file in JSON format and also to actually store the field values.

Configuration Files#

First, create a few files

logstash-simple.conf#

input {
  stdin {
    type => "stdin-type"
    format => "json"
  }

  file {
    debug => true
    format => "json"

    path => [ "/Users/username/prototype/logstash/*.log" ]
    
    start_position => "beginning"
    type => "file-type"
  }
}

output {
  stdout { 
    debug => true
    debug_format => "json"
  }

  elasticsearch { 
    embedded => true
  }
}

run-logstash.sh#

java -jar logstash-1.1.5-monolithic.jar agent -f logstash-complex.conf

run-logstash-web.sh#

java -jar logstash-1.1.5-monolithic.jar web --backend elasticsearch://localhost/

json.log#

This file can be any valid JSON file.

{"fname": "begin", "lname": "begin"}
{"fname": "david", "lname": "arcoleo"}
{"fname": "sarah", "lname": "arcoleo"}
{"fname": "karen", "lname": "arcoleo"}
{"fname": "joseph", "lname": "arcoleo"}
{"fname": "end", "lname": "end"}

Running & Testing#

In one shell, run

$ ./run_logstash.sh

In another do

$ ./run_logstash_web.sh

Wait until the java processes stop spiking and then, in another shell do

$ curl -s -XGET http://localhost:9200/_status\?pretty\=true

If you see

{
  "ok" : true,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "indices" : { }
}

then you don't have any data and something is wrong with your config. If you see anything else, you're good to go.

NOTE: If you see no data, you may have to tweak the JSON file. It seems to sometimes read only on file change. So just insert a blank line at top of the JSON file and save it (while the java processes are still running). You should see a bunch of output from your run-logstash.sh window.

Verifying#

Go to http://localhost:9292/ and put "*" in for the query (w/o the quotes). You should see every line in the JSON file.

Put in "fname:david" and you should just see the one line.