tent

Transsiberian, New Hoster, New Layout

As mentioned before I have been travelling for the last two months and could not find the time, and also no topic, to blog about. Together with a friend I took the Transsiberian railway from Beijing to Saint Petersburg. We spent a few days hiking the Great Baikal Trail with our tent, but it got horribly cold and after two freezing nights we dropped our camping plans. We got a quite good idea of the big wild Russia. It has been an awesome trip, but I’m also glad to be back home and inside my daily routine again.

Due to problems I had with my old hoster, including the limited access to configuration files and some security issues, I decided to move my blog to a new hoster. From now on my website resides at DigitalOcean who provide SSD cloud hosting. I chose a server farm in the Netherlands and I hope they will respect the European privacy laws there. Now I am in total control with ssh root access and I could configure a ssh public-private key authentication in almost no time. They charge you by every hour and will stop doing so as soon as your bill reaches a certain monthly limit (5$ in my case). Until now I can totally recommend it!

Furthermore I thought it was time to change the website’s layout again. New hoster, new layout. Kind of makes sense!

Website Downtime Due To WordPress Attack

Hello again everyone!

during the last two months I have been travelling a lot. I did only realize just now that my website went down due to an attack. Someone achieved to access my files either by the wordpress infrastructure or FTP. The attacker tried to put some malicious code at the beginning of all the PHP files, that would have been called on every access. “Sadly” he destroyed some of the files and that kept them and my whole website from working since then.

I think there might be other people affected by the same issue, that’s why I’d like to document the steps I took to help people with identical problems.

Bulk-remove the malicious code

My first approach was to remove the evil code snippet from all the files. I used www.regexpal.com and the linux tool called Kiki to figure out a regular expression that would find the snippet in some examples. You should use at least a few files to make sure it won’t find something else. Also use as many unique information as possible to avoid mistake with other parts of the code (pay attention: I realized that some parts of the snippet will change in every file, so you can’t search for these). The regex I came up with was

To find this snippet and replace it in every file you can move in the root folder of your wordpress installation and execute following linux command:

Revert WordPress files

This successfully removed the bad tag from all my files, but then I realized that some of the files were damaged at the last few lines (not from my actions, but due to the attack). So if this didn’t happen for you, you can now upload the fixed files and continue using your wordpress installation.

For me this obviously didn’t do the trick. I had to revert all my wordpress files. But I couldn’t remember the last version of wordpress that I installed. To figure out what wordpress version I had been using, I went to the folder wp-includes. There’s a file called version.php. The variable $wp_version will tell you which version to download. Extract the downloaded archive somewhere. Then copy all the contents from wp-content of your old fixed version to the newly extracted. That’s it! Don’t forget to change your passwords, so this will not happen again. Also make sure all plugins and wordpress itself have the latest version.

Quizduell Bot

Introduction

It all started when I was having breakfast with some friends. As it happens a lot recently we were playing Quizduell (for English readers: Quizduell is a very popular [at least in Germany] Android application produced by FEO Media). I can’t remember the exact story but eventually one of them said “I bet you won’t be able to create a computer program that will be able to beat me in Quizduell”. Those words kind of hurt our digital hearts deep inside, that’s why we accepted his bet. We had four weeks to work on our bot before he would need to compete against this (former) friend of ours.

The basic idea we developed

How to interact with the game

First we thought about using their webservices directly, interacting with them like the normal application. We realized very quickly that this would cost us a lot of time. I was still writing exams by then and didn’t have too much of it (time). So we decided to use the application that existed already and control it from an extern point. Google is providing some tool called Monkeyrunner which can be used to control a device/application from an USB connection. But still we would need some way to see what the screen was currently showing. Monkeyrunner offers the possibility to take screenshots. We thought about using some OCR software to analyze these pictures. In the end we found another/faster way to do it. Diego Torres wrote a python library called AndroidViewClient which made life much easier for us. His code let us inspect the window/view stack that the device is currently showing. Each view has some properties (not all of them are configured with the correct values) that can be examined to detect where to click or what to do.

Our final solution:

  1. First we’re asking the device which activity it is executing. (We might start Quizduell here if it is not running yet)
  2. After that we will use the activity to determine in what program section we currently are. This is not very specific, there are two big ones: GameTableActivity, GameActivity.
  3. Now we will use AndroidViewClient to get the current view stack from android’s window manager.
  4. GameActivity is split into different smaller screens. One where the game overview is shown, one for the question and its answers, one to choose category …
    These will be detected by specific view elements that should be there or that should have a precise and deterministic value.
  5. By then we will know where we currently are and we will be able to make a choice where to put our finger.

Answering algorithm

Another question that we asked ourselves was how to find the correct answers for the questions that would come up.

One possibility was to just send four requests to Google, each one with the question and one of the answers. We would take the one answer with the most search results for the correct one. This didn’t work quite well. For instance if a question should be negated, then we would need to take the answer with least search results and so on. Not to mention the difficulties we had with Google blocking our search requests (because we weren’t using a normal browser).

Another strategy would have been to use the database www.quizduell-loesung.at (which had still been online at this point in time). It offered answers to a lot of questions that the program was asking.

My flatmate found this article explaining how to reverse engineer the application. I tried to figure out a way to achieve something like he did (I got interested because I never had the chance to reverse engineer for something reasonable). In the end I had a comparable approach. Instead of using the string to find the responsible view I had been using the green background of the correct button to find the location within the code that would assign this image to a view. Finally I set the content-description of the buttons that contained the answers with either “wrong” or “correct”. Now when we would use the AndroidViewClient to request the view stack and the question with its answer buttons would be detected we could just click on the view that had the content-description equal to “correct”.

Result

We were able to restore our dignity … and got free drinks and cinema. :)

Where’s the code?

I would like to put a download link here.

Quizduell’s EULA states:

With regard to the competitive functions of the App, you must not use the App in any way that could be deemed dishonest or unfair in order to gain an advantage over other users.

This is why I am afraid our approach was not that legal and I may not encourage any other person to do alike.

At this point I would like to make clear that we’re not interested in ruling the Quizduell charts. This was a pure scientific approach :)

Video

Arduino Alarm Clock

Finally I have managed to finish my Arduino Alarm Clock with Google Calendar support. I used PiTiVi to create a video on what it is capable to do.

If you don’t wish to watch the full video, these are the most important features:

  • The cube is able to glow in all kinds of colors.
  • It is connected to the wifi and fetches current time, weather forecast and events form my Google Calendar periodically.
  • Supports different modes:
    • Clock
      • Provides a static overview on time and events.
      • Glows more the nearer an event gets.
    • Alarm
      • Will wake you a configurable amount of minutes before the event.
      • As soon as event is reached it plays nokia ringtones and flashes in different colors.
      • Tapping on top of the cube will stop the alarm.
    • Weather
      • Displays detailed weather forecast for today.
      • Glows more the nearer an event gets.
    • Demo
      • Switches between easy to read pages, showing time, upcoming event and current weather.
      • Glows more the nearer an event gets.
  • The user is able to switch between modes by tapping on the top of the cube.
Screenshot-from-2013-10-31-135815

Installing PiTiVi on Fedora 19

Ever tried to do video editing on Linux?
I have several times, and mostly it’s been about avoiding mistakes that would crash the underlying application.

Odyssey of Installing PiTiVi

Just yesterday I have discovered a new program called PiTiVi (www.pitivi.org). The clean interface that the screenshots on their website were showing made me go and give it a try. But as I was installing the latest version from the Fedora repositories I realized that the maintained version was quite old (0.15.2). The current up-to-date version is 0.91, although this is the direct successor of 0.15 there has been lot of changes and improvements that I did not want to miss. Of course I gave 0.15 a first try and it made a good impression: sleek design, clean interface, easy to use, stable. Only the preview window was quite slow and began to chop when I added more than one clip to the stage. So there was at least one reason to update to the latest version.

The website tells to just get the code and compile. But what they only tell in the development section: the latest PiTiVi release uses a different architecture then previous releases. This is why you’ll need the GStreamer Editing Services which have not been build for Fedora 19 yet … But then there is a wiki page explaining how to build these dependencies on Fedora. I followed those steps and after a few attempts finally got it to work.

Easy Installation

To (well, hopefully it works out) save you some time and trouble I wrote two little scripts that help you with installing and running the new PiTiVi 0.91 on Fedora.

Therefor I used two scripts from PiTiVi Wiki as well. Firstly the script which can be found here: http://git.gnome.org/browse/pitivi/plain/bin/pitivi-git-environment.sh (which I have called install_ges_for_pitivi.sh) and secondly parts of this listing of all dependencies: http://wiki.pitivi.org/wiki/Dependencies.

Feel free to download Pitivi4Fedora here: Pitivi4Fedora.tar
Extract it to a destination of your choice. Move inside and execute (in a terminal):
sh ./install_pitivi.sh
After that (you should see some ASCII art of a sailing duck at the end) you can run (with or without terminal):
sh ./run_pitivi.sh

Very Fast Screen Capture With Windows 8, C# and DirectX11

Lately I have experienced problems while using my selfmade Ambilight.
This Ambilight consists of LED-strips that are clamped to the back of my monitor, an Arduino as control unit and finally a tool running on the OS to measure and transmit the colors that currently are displayed on the screen. The program ran more or less OK on Windows 7 – just for your information, until today I have been using the DirectX 9 approach which you can read about here: http://www.codeproject.com/Articles/5051/
Various-methods-for-capturing-the-screen
. Whenever the program was slowing down my computer too much I would disable DWM (don’t know why it performed better without the new DWM, guess it was just not so well integrated at that time).

Now, after moving city, I installed the Ambilight to my computer monitor again. I didn’t only change residence, but also OS. I have been shocked to find out that with Windows 8 DWM can’t be turned off anymore. So lately my Ambilight became more a pain in the ass then it was entertaining really. I spent the last two days surfing the web and searching for a new/better way of capturing images of the desktop (and maybe even fullscreen games etc).

Eventually I discovered some articles about a new DXGI version (1.2) that came out with Windows 8 and which introduced Desktop Duplication.

My current program was written in C# and used the SlimDX wrapper to access the DirectX API. SlimDX sadly does not support version 1.2 of the DXG Infrastructure yet, that’s why I moved to SharpDX. Pay attention to reference the libraries from the “Win8Desktop-net40″ folder, otherwise you won’t have the new DXGI interface.

Below you will find a little test program that I wrote to test performance of this new approach:

Update:

I will just summarize the main points that arose from the comments below (and some others that I figured out during the last few days):

  • Although MSDN states that at least Windows 7 with Platform Update is required to use the DesktopDuplication, this is not true. The Platform Update will just follow the interface convention of DXGI 1.2 but won’t implement the newly added methods. (instead they will just fail with error code E_NOTIMPL.
  • I haven’t found a very efficient way to save the SharpDX.DataStream yet. I mixed up some code from Stackoverflow that will help you to transform the stream into a Bitmap object which can be saved easily.
  • Fullscreen DirectX applications can not be captured without source code control. If the SwapChain hasn’t been created with the right flags, there is (at least without any system-hacking-skills) no way to duplicate the screen. You can find more information about that on the SwapChain Flags MSDN page. Now as I see it you will either have to use IDXGIFactory2::CreateSwapChainForHwnd, IDXGIFactory2::CreateSwapChainForCoreWindow or IDXGIFactory2::CreateSwapChainForComposition without the DXGI_SWAP_CHAIN_FLAG_DISPLAY_ONLY to allow other applications to access the SwapChain’s backbuffer.
DSC02001

WiFi Library for chipKIT Uno32

No I didn’t forget about the WiFi Clock Tutorial … In fact I am currently still working on the clock. My latest improvements broke the LC-display (I guess you wouldn’t call this improvements, right?). Since then I was ordering different stuff on the internet.

First I gave up the LCD for a brand new OLED, which I got from www.adafruit.com. It is only black and white (and also smaller than my old one), but the contrast is just amazing :) .

Next I realized that this would grow my program’s heap memory (the display’s content needs to be buffered in the RAM) about another 400-500 bytes. This wouldn’t fit into the Arduino Uno’s little 2k SRAM anymore.
I then had two options:
a) buy an SPI SRAM module or
b) buy a new board with sufficient SRAM
I went for the second option and after some comparison of existing manufacturers I bought the chipKIT Uno32. Mainly because its manufacturer pays a lot attention to maintain Arduino compability.

Once the Uno32 had arrived I tried and plugged the WiShield 3.0 onto it (elechouse might be the fabricator). It fit well, I didn’t expect anything different because the board’s dimensions are the same as the Arduino’s. But it would not work without any adjustments. Allthough the two products have the same size their pin layout differs in some cases. So i could assemble board and shield, but the library just wouldn’t compile anymore with the new Uno32 IDE (basically the same frontend with different compiler, the Uno32 does have a pic32 instead of an avr-processor).

So long story short, in the last weeks I have adapted the WiShield-library to match the new pin-layout and also to use the specific registers for SPI communication and interrupt usage.
If you want to use my revision, this link will let you download the latest version: WiShield Library for Uno32.

Mastermind

One of the semester’s courses I have been attending was about SmartCards and security. We had one excercise about implementing MasterMind on our cards. In the end I knew there was something wrong with my calculation of white and black markers. Since debugging on the card is kind of difficult I began writing Java code in Eclipse. At the beginning I only rewrote that simple algorithm to calculate the black and white markers for a given guess and the secret code. But as time went by my program grew. Eventually I did implement Donald Kunth’s five-guess algorithm which uses the Minimax decision rule to find the “best worst case” for a given scenario. This way it will always solve puzzles in five steps and less. It is a really interesting concept which I haven’t heard of before. If you are interested in game theory and/or mathematics, it is a must read.

So finally the program is done and I think the algorithms that I have been implementing are not that bad. I have used another program with similar features that I found using “a search machine of my choice”  to compare performance. Results showed that my version could solve a few times more puzzles in the same time (depends on hardware, on my machine up to 8x). But since the author wanted to show how fast its CUDA implementation was in comparison, I think he didn’t lay too much emphasis on creating a fast C# solution.

Anyways if you want to use this MasterMind-cheating-tool, feel free to follow the download link below. Because it is written in C# you will need at least .Net 4.0. Also it is a console application, so don’t expect too much shininess. Currently it features a little benchmark and the possibility to solve a puzzle interactively (which means, that it will tell you which combinations to try after looking at the white and black marker count that you typed in).

Download the source of MasterMind Solver right here.
Or go for the binary only.

Website Update And New Photos

[singlepic id=226 w=320 h=240 float=left] I finally managed to create a new website header. Might keep this one for a while, we will see :-) .
Also I rewrote the references section – should be more international now.

Ah yes and I uploaded some pictures of my trip to the baltics. I drove about 4000 Km and visited Poland, Lithuania, Latvia, Estonia, Sweden and Denmark (and these are even ordered chronologically). Hope you enjoy them!

PS: I am really looking forward to release the howto on making your own Google Calendar featured alarm clock. I began writing the first parts, but then got suck into exams. You will have it as soon as possible – promise.