Talk on Infrastructure at Scale: Apache Kafka, Storm and Elastic Search

The following talk gives a good overview on the technologies used by Loggly.com to build their scalable real-time-search log-processing infrastructure.

The slides of the talk can be found here:

Infrastructure at Scale: Apache Kafka, Twitter Storm & Elastic Search (ARC303) | AWS re:Invent 2013 from Amazon Web Services

Stanford Large Network Dataset Collection

Stanford Large Network Dataset Collection
Stanford Large Network Dataset Collection

The Stanford Large Network Dataset Collection was published as part of the Stanford Network Analysis Project (SNAP). It consists of an interesting collection of large networks. The aim of SNAP is provide a general purpose network analysis and graph mining library.

The SNAP Collection contains datasets from various domains such as:

It is definitely worth of having a look at the different datasets. What dataset are you missing? What would you like to be added to the collection? Leave a comment below!

(via Hacker News)

Enable HTTPS everywhere

SSL-Report for blog.muehlburger.at
SSL-Report for blog.muehlburger.at

Finally I finished to configure my HTTPS protocol support for blog.muehlburger.at. Supporting encrypted communication is an important part of the internet today. Everybody should support encrypted communication on the web. A good resource on how to select strong cypher suites and to configure your web server to support encryption properly is bettercrypto.org.

There is also a great video covering the current state of the art in crypto held by security researchers at 30C3:

I configured nginx to support encrypted communication exclusively for my WordPress installation. The PDF guide from bettercrypto.org was a great resource for selecting the cypher suits and some additional parameters.

If you would like to configure your nginx webserver and WordPress installation to support https just drop me a message. I am pleased to help you with my experiences.

(via A year in Crypto)

How to fix error on Pentaho Data Integration (Kettle) startup

Howto fix startup error of Pentaho Data Integration (Kettle) on CentOS 6? You just need to modify the spoon.sh startup script after downloading and unzipping Pentaho Data Integration. The modification updates the Java runtime options for Kettle to startup properly. Therefore open

[code language=”bash”]spoon.sh[/code]

and change the end of the file in the following way:

[code language=”bash”]
# ******************************************************************
# ** Set java runtime options **
# ** Change 512m to higher values in case you run out of memory **
# ** or set the PENTAHO_DI_JAVA_OPTIONS environment variable **
# ******************************************************************

if [ -z "$PENTAHO_DI_JAVA_OPTIONS" ]; then
PENTAHO_DI_JAVA_OPTIONS="-Xmx512m -XX:MaxPermSize=256m"
fi

OPT="$OPT $PENTAHO_DI_JAVA_OPTIONS -Djava.library.path=$LIBPATH -DKETTLE_HOME=$KETTLE_HOME -DKETTLE_REPOSITORY=$KETTLE_REPOSITORY -DKETTLE_USER=$KETTLE_USER -DKETTLE_PASSWORD=$KETTLE_PASSWORD -DKETTLE_PLUGIN_PACKAGES=$KETTLE_PLUGIN_PACKAGES -DKETTLE_LOG_SIZE_LIMIT=$KETTLE_LOG_SIZE_LIMIT -Dorg.eclipse.swt.browser.XULRunnerPath=/dev/null"
# ***************
# ** Run… **
# ***************
"$_PENTAHO_JAVA" $OPT $STARTUP -lib $LIBPATH "${1+$@}"
[/code]

Everything I did was appending

[code language=”bash”]-Dorg.eclipse.swt.browser.XULRunnerPath=/dev/null"[/code]

to the

[code language=”bash”]OPT[/code]

part as described in ticket TDI-24139.

Javapocalypse

I need to share the following video on “Javapocalypse” from JavaZone. Pushing the “Undo Java” button will have huge impact. Have a look at these consequences. The following trailer highlights the importance of Java.

JavaZone 2013 takes places in Oslo at the 10th and 11th of September 2013.

JavaZone is an important meeting place for software developers in Scandinavia, and is the biggest community driven conference of its kind. JavaZone has been described as a high quality, independent conference – a leading forum for knowledge exchange for IT- professionals. All speakers are selected by the independent Program Committee. (via jz13)

What do you think about the future of the Java stack?

(via jz13)

What Most Schools Don’t Teach – Programming

“The programmers of tomorrow are the wizards of the future. You are going to look like you have magic power, compared to everybody else.” Diese Aussage beschreibt den Wert, den “Coding Skills” heute und in Zukunft haben. Die Fähigkeit Software nach eigenen Vorstellungen zu entwickeln stellt für jeden Bürger eine wichtige Möglichkeit dar das Grundrecht der Freiheit auszuleben.

Das folgende Video beschreibt was “die meisten Schulen (noch) nicht lehren” nämlich Software zu entwickeln und Quellcode lesen zu können.

Meine ersten Schritte der Programmierung mit habe ich im Alter von vierzehn Jahren mit den Programmiersprachen Java und PHP gemacht. Wann habt ihr mit dem Programmieren begonnen und was waren eure ersten Erlebnisse?

(via code.org)

Video Introduction to GIT Version Control by Scott Chacon of GitHub

The following talk, given by Scott Chacon of GitHub, gives a very good introduction to the version control system GIT and its usage. This is by far the best introduction that I have found so far.

(via Marakana)

Simple Flashlight v3.1

Simple Flashlight for Android

Die “Simple Flashlight” App für Android ist nun in der Version 3.1 im Market verfügbar. Simple Flashlight ist eine einfache Taschenlampe für dein Mobiltelefon oder deinen Tablet. Es wurde die transparente Displaybeleuchtung entfernt. Zudem gibt es jetzt die Möglichkeit, durch “Wischen” nach rechts bzw. links die Farbe des Bildschirms auf rot oder blau zu ändern.

Recent Changes (v3.1)

  • Displaybeleuchtung verbessert (Transparenz entfernt)
  • Swipe nach rechts => Bildschirm blau färben
  • Swipe nach links => Bildschirm rot färben
  • Swipe verbessert

Recent Changes (v3.0)

  •  Möglichkeit die App zu starten indem man die Suchtaste gedrückt hält
  •  Bildschirmbeleuchtung ist nun transparent
  •  Swipe-up und Swipe-Down zum dimmen der Bildschirmhelligkeit

Simple Flashlight im Android Market

Downloaden und ausprobieren! https://market.android.com/details?id=at.muehlburger.android.simple.flashlight

Simple Flashlight v3.0

Simple Flashlight for Android

Die “Simple Flashlight” App für Android ist nun in der Version 3.0 im Market verfügbar. Simple Flashlight ist eine einfache Taschenlampe für dein Mobiltelefon oder deinen Tablet.

Recent Changes (v3.0)

  •  Möglichkeit die App zu starten indem man die Suchtaste gedrückt hält
  •  Bildschirmbeleuchtung ist nun transparent
  •  Swipe-up und Swipe-Down zum dimmen der Bildschirmhelligkeit

Simple Flashlight im Android Market

Downloaden und ausprobieren: https://market.android.com/details?id=at.muehlburger.android.simple.flashlight

Enhanced by Zemanta

Machine Learning is awesome

Machine Learning Class

Machine Learning has always been a topic that attracted my interest. Currently I attend the “Machine Learning” class (@ml_class) offered by Prof. Andrew Ng from Stanford University. The class is awesome. The video lectures are made of small chunks explaining every topic in detail. A lot of topics were already covered:

  • Linear regression with one variable
  • Linear regression with multiple variable
  • One-vs-all Classification
  • Regularization
  • Backpropagation Algorithm
  • Neural Networks
  • Practical advise for applying learning algorithms
  • How to develop and debug learning algorithms
  • Feature and model design, setting up experiments

Other interesting topics are following. In parallel to the lectures there are homework programming exercises that have to be solved. To date programming assignments covering topics as:

  • Linear regression
  • Logistic regression
  • Multi-class classification and Neural Networks
  • Neural network learning
  • Regularized linear regression and bias-variance

In order to solve the exercises you have to understand the contents and have some programming experience in GNU Octave. It is also important to have basic understanding of Linear Algebra.

Machine Learning Contests

In order to apply machine learning on real world problems you can enter a machine learning contest. There are various contests out there, some of which are

I will keep you updated.

Enhanced by Zemanta