Nov 10, 2013

PhpSysInfo loading slow on RaspberryPI (Linux Raspbian + Nginx + PHP5-fpm)

Default installation of PhpSysInfo on RaspberryPI takes a long time to load due some missing programs and hardware components.

My system info:


Hardware: RaspberryPI Model B with 512MB RAM
OS: Raspbian GNU/Linux 7
Nginx version: 1.2.1-2.2+wheezy1
PHP version: 5.4.4-14+deb7u5
PhpSysInfo version: 3.0.17-1

Enable debug mode in /etc/phpsysinfo/config.php

define('PSI_DEBUG', true);

Open http://yoursitephpsysinfo/xml.php

I found the following errors:

<Error Function="find_program(lsb_release)">
<![CDATA[
program not found on the machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 576 in function build() ./includes/os/class.Linux.inc.php on line 527 in function _distro() ./includes/class.CommonFunctions.inc.php on line 117 in function executeProgram( "lsb_release", "-a 2>/dev/null", "", true )
]]>
</Error>
<Error Function="/usr/bin/lspci">
<![CDATA[
pcilib: Cannot open /proc/bus/pci lspci: Cannot find any working access method. Return value: 1 ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 583 in function build() ./includes/os/class.Linux.inc.php on line 306 in function _pci() ./includes/class.Parser.inc.php on line 36 in function lspci() ./includes/class.CommonFunctions.inc.php on line 142 in function executeProgram( "lspci", "", "", true )
]]>
</Error>
<Error Function="find_program(lsscsi)">
<![CDATA[
program not found on the machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 585 in function build() ./includes/os/class.Linux.inc.php on line 367 in function _scsi() ./includes/class.CommonFunctions.inc.php on line 117 in function executeProgram( "lsscsi", "-c", "", true )
]]>
</Error>
<Error Function="file_exists(/proc/scsi/scsi)">
<![CDATA[
the file does not exist on your machine ./xml.php on line 45 ./includes/output/class.WebpageXML.inc.php on line 138 in function run() ./includes/xml/class.XML.inc.php on line 456 in function getXml() ./includes/xml/class.XML.inc.php on line 435 in function _buildXml() ./includes/os/class.OS.inc.php on line 70 in function getSys() ./includes/os/class.Linux.inc.php on line 585 in function build() ./includes/os/class.Linux.inc.php on line 367 in function _scsi() ./includes/class.CommonFunctions.inc.php on line 191 in function rfts( "/proc/scsi/scsi", "", 0, 4096, true )
]]>
</Error>

First, third and fourth error are because some programs are not present on the system, let's install them.

sudo apt-get install lsb-release lsscsi -y

Second error is due missing PCI bus on raspberryPI.

# lspci
pcilib: Cannot open /proc/bus/pci
lspci: Cannot find any working access method.

We can't do anything about that but disable that function in PhpSysInfo PHP code. Open the file /your/path/to/phpsyinfo/includes/class.Parser.inc.php and find the lspci function.

/**
     * parsing the output of lspci command
     *
     * @return Array
     */
    public static function lspci()
        {
        $arrResults = array();
        if (CommonFunctions::executeProgram("lspci", "", $strBuf, PSI_DEBUG)) {
            $arrLines = preg_split("/\n/", $strBuf, -1, PREG_SPLIT_NO_EMPTY);
            foreach ($arrLines as $strLine) {
                list($strAddr, $strName) = preg_split('/ /', trim($strLine), 2);
                $strName = preg_replace('/\(.*\)/', '', $strName);
                $dev = new HWDevice();
                $dev->setName($strName);
                $arrResults[] = $dev;
            }
        }
        return $arrResults;
    }

Change the code above to match the one bellow:

/**
     * parsing the output of lspci command
     *
     * @return Array
     */
    public static function lspci()
        {
        return array();
            $arrResults = array();
        if (CommonFunctions::executeProgram("lspci", "", $strBuf, PSI_DEBUG)) {
            $arrLines = preg_split("/\n/", $strBuf, -1, PREG_SPLIT_NO_EMPTY);
            foreach ($arrLines as $strLine) {
                list($strAddr, $strName) = preg_split('/ /', trim($strLine), 2);
                $strName = preg_replace('/\(.*\)/', '', $strName);
                $dev = new HWDevice();
                $dev->setName($strName);
                $arrResults[] = $dev;
            }
        }
        return $arrResults;
    }

Disable debug mode in phpsysinfo config, save, reload, drink beer.

Nov 8, 2013

Ubuntu Fail2ban fails to parse Apache access.log

I was trying to setup Fail2ban to block Wordpress login bruteforce attacks, but Fail2ban somehow failed to parse access.log

When parsing log file with command:
fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-wp-login.conf
CPU rises to 100% usage until I kill the process. I made a quick fix - workaround with redirecting needed content from access.log to another log file.

My setup is as follows:

OS: Ubuntu Server 12.04 LTS
Fail2ban: 0.8.6-3wheezy2build0.12.04.1
Python: 2.7.3-0ubuntu2.2

Fail2ban configuration:

/etc/fail2ban/fail2ban.conf
# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
# $Revision$
#
[Definition]
# Option:  loglevel
# Notes.:  Set the log level output.
#          1 = ERROR
#          2 = WARN
#          3 = INFO
#          4 = DEBUG
# Values:  NUM  Default:  3
#
loglevel = 4
# Option:  logtarget
# Notes.:  Set the log target. This could be a file, SYSLOG, STDERR or STDOUT.
#          Only one log target can be specified.
# Values:  STDOUT STDERR SYSLOG file  Default:  /var/log/fail2ban.log
#
logtarget = /var/log/fail2ban.log
# Option: socket
# Notes.: Set the socket file. This is used to communicate with the daemon. Do
#         not remove this file when Fail2ban runs. It will not be possible to
#         communicate with the server afterwards.
# Values: FILE  Default:  /var/run/fail2ban/fail2ban.sock
#
socket = /var/run/fail2ban/fail2ban.sock
We add new entry for Wordpress

Add configuration to /etc/fail2ban/jail.conf
[apache-wp-login]
enabled  = true
port     = http,https
filter   = apache-wp-login
logpath  = /var/log/apache2/apache-wp-login.log
maxretry = 3
findtime = 60
Now me make new filter for Wordpress. Some servers logs are in different format so we have to make different regex entries. These are two most common configurations:

Log format example 1:
www.domain.si:80 188.65.115.90 - - [08/Nov/2013:13:20:46 +0100] "POST /en/wp-login.php HTTP/1.1" 200 1784 "http://www.domain.si/en/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36"
Log format example 2:
89.222.15.152 - - [08/Nov/2013:13:40:12 +0000] "POST /wp-login.php HTTP/1.1" 200 1756 "http://domain.wordpress.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/25.0"
Create /etc/fail2ban/filter.d/apache-wp-login.conf

For log format example 1 use:
[Definition]
failregex = ^[^\:]+\:80 <HOST> .* "POST
ignoreregex =
ATTENTION! Don't apply this filter to default access.log, it would block all POST request, not only Wordpress. Only use it with modified log file we will create bellow.

For log format example 2 use:
failregex = <HOST>.*] "POST /wp-login.php
ATTENTION! Don't apply this filter to log format example 1, it would block your own server, because <HOST> regex would match your domain (www.domain.si).

Now we create new log file for parsing (/var/log/apache2/apache-wp-login.log). We only want to filter out POST requests for wp-login.php and write them to new log file called apache-wp-login.log.
tail --follow=name /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &
Add this command to /etc/rc.local so it would run at reboot:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Fail2ban Wordpress Login
tail -f /var/log/apache2/access.log | grep --line-buffered wp-login.php > /var/log/apache2/apache-wp-login.log &

exit 0
Restart Fail2ban and you're done.
/etc/init.d/fail2ban restart






Oct 10, 2013

Wordpress update - old jQuery warnings

With the recent Wordpress update, jQuery was also updated to version 1.10.

Blogs were now full of warning messages, somewhere along the lines of:

ATTENTION! (by Comprehensive Google Map Plugin)
Your blog/site theme or one of your plugins uses jQuery javascript library which is older than the version 1.3.0.
The Comprehensive Google Map plugin will not work with such outdated jQuery version.
The minimum jQuery requirement for Comprehensive Google Map plugin is version 1.3.0. Apologies for the inconvenience..

However, this is not true. Version 1.10.2 is greater than 1.3, but somehow it get's detected as 1.1, because silly programmers cast the version code into float. Ugh.

if (version < 1.3) { // WTF ?!?!?!?!?!
    alert(CGMPGlobal.errors.oldJquery);
    return false;
}
Unfortunately, there's not much we can do about it, other than a quick hack of removing the version check or modifying it to 1.1.

Or wait for the plugin developers to FIX THE DAMN CODE.

Like that will happen anytime soon.


Oct 5, 2013

Restore panoramas (cubic tiles) from exported Pano2VR

With this simple little script you can restore the full size cubic faces of the panorama which was exported using Pano2VR.

Pretty useful, if you deleted the original by mistake ;)

To convert the cubic projection back to equirectangular, you can also use Pano2VR, but for the input source you choose "Cubic", load all six cube faces, and export transformation.


Pano2VR's panoramas are exported as tiles in a cubic projection. The file naming scheme is as follows:
c[CUBE FACE]_l[RESOLUTION LEVEL]_[X]_[Y].jpg
for example:
c0_l0_0_0.jpg.

The most detailed resolution level is "0", so that level is used, and other tiles are disregarded.
Cube faces range from 0 to 5, and X and Y are the row and column number.

The script is written in PHP, and should be run in a CLI.

error_reporting(E_ALL);
ini_set('display_errors', 1);
ini_set('memory_limit', '2G');
date_default_timezone_set('Europe/Ljubljana');

$dir = 'tiles/';

foreach(range(0,5) as $c) {
 $i = 0;
 $tiles = array();
 $height = 0;
 do {
  $width = 0;
  $j = 0;
  do {
   $fn = fname($c,$i,$j);
   if(file_exists($fn)) {
    $tiles[$i][$j] = $fn;
    list($w, $h) = getimagesize($fn);
    $width += $w;
   }
   $j++;
  } while(file_exists(fname($c,$i,$j)));
  $j--;
  $i++;
  $height += $h;
 } while(file_exists(fname($c,$i,$j)));
 $i--;
 
 echo $width.' '.$height."\n";
 $im = imagecreatetruecolor($width, $height);
 $y = 0;
 foreach($tiles as $row) {
  $x = 0;
  foreach($row as $tile) {
   $tile = imagecreatefromjpeg($tile);
   $src_w = imagesx($tile);
   $src_h = imagesy($tile);
   imagecopy($im, $tile, $x, $y, 0, 0, $src_w, $src_h);
   $x += $src_w;
   imagedestroy($tile);
  }
  $y += $src_h;
 }
 imagejpeg($im, "$c.jpg", 70);
}

function fname($c,$i,$j) {
 global $dir;
 return "{$dir}c{$c}_l0_{$i}_{$j}.jpg";
}

Tiles

Restored cube faces

Restored panorama 



Sep 13, 2013

Embedding flash in Facebook link posts

If you want to have custom flash movies embeded in Facebook posts that link to your website, like these one from YouTube or SoundCloud:

 

It's fairly simple. 
BUT, it requires a SSL certificate, so that your website works over HTTPS, without browsers whining about security.

Here's what you do.
You've heard of OpenGraph protocol, right? Right. So, that's all there is to it. Proper metadata in the HTML of your website will instruct Facebook to include your flash movie, like this:
<meta property="og:title" content="Aljažev stolp | Triglav (2864m) | VR panorama"/>
<meta property="og:video" content="http://kafol.net/pano/aljazev-stolp/aljazev.stolp.swf"/> 
<meta property="og:type" content="movie"/>
<meta property="og:url" content="http://kafol.net/pano/aljazev-stolp/"/>
<meta property="og:image" content="http://kafol.net/pano/aljazev-stolp/preview.jpg"/>
<meta property="og:description" content="VR (virtual reality) panorama pri Aljaževem stolpu na vrhu Triglava. Panorama je posneta brez stativa."/>
<meta property="og:video:type" content="application/x-shockwave-flash" />
<meta property="og:video:width" content="400" />
<meta property="og:video:height" content="300" />


The next part depends on what the flash movie actually does. If it's a movie/music player that fetches media from other sources, or even an other part of your website, you'll need to configure crossdomain.xml. You put that file on the root of your website.

Here's a very liberal example, but fine for testing:
<?xml version="1.0" ?>
<cross-domain-policy>
<allow-access-from domain="*" />
</cross-domain-policy>