Quad Core UDOO – Debian – compiling kernel 4.0.8

U-Boot 2015 http://www.igorpecovnik.com/2015/02/20/udoo-quad-debian-sd-image/

Download the software:

wget http://mirror.igorpecovnik.com/kernel/4.0.8-udoo-next.tar

extract it and only install the u-boot package:

dpkg -i linux-u-boot-4.0.8-udoo_1.8_armhf.deb

 

Kernel based on https://github.com/patrykk/linux-udoo

Install prerequisites:

apt-get install lzop kernel-package u-boot-tools dblatex docbook-utils \
 kernel-common libsgmls-perl libsp1c2 libtinfo-dev sgmlspl sp texlive \
 texlive-bibtex-extra texlive-math-extra xmlto xsltproc libncurses5-dev

Getting the kernel:

cd /tmp
mkdir kernel4.0.8
cd kernel4.0.8
wget https://github.com/patrykk/linux-udoo/archive/4.0.8.zip
unzip 4.0.8.zip

now change directory to the kernel folder and configure it:

make menuconfig

compile it – in my case it took approx 2 hours:

make uImage LOADADDR=0x10008000  modules

install the modules:

make modules_install

copy the kernel to /boot folder

cp arch/arm/boot/uImage /boot/
 cp arch/arm/boot/zImage /boot/

compile the dtb file:

make imx6q-udoo.dtb

copy dtb file
cp arch/arm/boot/dts/imx6q-udoo.dtb /boot/dts/
copy firmware
cp firmware/imx/sdma/sdma-imx6q.bin /lib/firmware/sdma/
configure your uboot
nano /boot/boot.cmd
then copy and paste this
setenv bootargs root=/dev/mmcblk0p1 rootfstype=ext4 rootwait console=ttymxc1,115200
ext2load mmc 0 0x18000000 /boot/dtb/imx6q-udoo.dtb
ext2load mmc 0 0x12000000 /boot/zImage
bootz 0x12000000 - 0x18000000

if you use graphical interface you will probably want to replace the first line with:

setenv bootargs root=/dev/mmcblk0p1 rootfstype=ext4 rootwait console=tty1 video=mxcfb0:dev=hdmi,1920x1080M@60,if=RGB24,bpp=32 ahci_imx.hotplug=1 quiet

then convert it to uboot format
mkimage -C none -A arm -T script -d /boot/boot.cmd /boot/boot.scr

Now cross fingers, connect console cable and restart the device

 

U-Boot 2015.04-dirty (Jul 13 2015 - 12:27:45)
CPU:   Freescale i.MX6Q rev1.2 at 792 MHz
 Reset cause: WDOG
 Board: Udoo
 DRAM:  1 GiB
 MMC:   FSL_SDHC: 0
 *** Warning - bad CRC, using default environment
In:    serial
 Out:   serial
 Err:   serial
 Net:   using phy at 6
 FEC [PRIME]
 Hit any key to stop autoboot:  0
 switch to partitions #0, OK
 mmc0 is current device
 295 bytes read in 63 ms (3.9 KiB/s)
 Running bootscript from mmc ...
 ## Executing script at 12000000
 35681 bytes read in 138 ms (252 KiB/s)
 6491688 bytes read in 374 ms (16.6 MiB/s)
 Kernel image @ 0x12000000 [ 0x000000 - 0x630e28 ]
 ## Flattened Device Tree blob at 18000000
 Booting using the fdt blob at 0x18000000
 Using Device Tree in place at 18000000, end 1800bb60
Starting kernel ...
Booting Linux on physical CPU 0x0
 Linux version 4.0.8 (root@UDOO) (gcc version 4.9.2 (Debian 4.9.2-10) ) #3 SMP Fri
 Jul 31 19:05:40 BST 2015
 CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
 CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
 Machine model: Udoo i.MX6 Quad Board
 cma: Reserved 256 MiB at 0x40000000
 Memory policy: Data cache writealloc
 PERCPU: Embedded 11 pages/cpu @af731000 s12480 r8192 d24384 u45056
 Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 260096
 Kernel command line: root=/dev/mmcblk0p1 rootfstype=ext4 rootwait console=ttymxc
 1,115200 ahci_imx.hotplug=1
 PID hash table entries: 4096 (order: 2, 16384 bytes)
 Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
 Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
 Memory: 764764K/1048576K available (8251K kernel code, 298K rwdata, 2472K rodata
 , 352K init, 440K bss, 21668K reserved, 262144K cma-reserved, 0K highmem)
 Virtual kernel memory layout:
 vector  : 0xffff0000 - 0xffff1000   (   4 kB)
 fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
 vmalloc : 0xc0800000 - 0xff000000   (1000 MB)
 lowmem  : 0x80000000 - 0xc0000000   (1024 MB)
 pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)
 modules : 0x7f000000 - 0x7fe00000   (  14 MB)
 .text : 0x80008000 - 0x80a811a8   (10725 kB)
 .init : 0x80a82000 - 0x80ada000   ( 352 kB)
 .data : 0x80ada000 - 0x80b24860   ( 299 kB)
 .bss : 0x80b24860 - 0x80b92b48   ( 441 kB)
 SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
 Hierarchical RCU implementation.
 NR_IRQS:16 nr_irqs:16 16
 L2C-310 erratum 769419 enabled
 L2C-310 enabling early BRESP for Cortex-A9
 L2C-310 full line of zeros enabled for Cortex-A9
 L2C-310 ID prefetch enabled, offset 1 lines
 L2C-310 dynamic clock gating enabled, standby mode enabled
 L2C-310 cache controller enabled, 16 ways, 1024 kB
 L2C-310: CACHE_ID 0x410000c7, AUX_CTRL 0x76070001
 VPU 352M is enabled!
 Switching to timer-based delay loop, resolution 333ns
 sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 1431655765682ns
 Console: colour dummy device 80x30
 Calibrating delay loop (skipped), value calculated using timer frequency.. 6.00
 BogoMIPS (lpj=30000)
 pid_max: default: 32768 minimum: 301
 Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
 Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
 Initializing cgroup subsys net_cls
 CPU: Testing write buffer coherency: ok
 CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
 Setting up static identity map for 0x107da3e8 - 0x107da440
 CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
 CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
 CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
 Brought up 4 CPUs
 SMP: Total of 4 processors activated (24.00 BogoMIPS).
 CPU: All CPU(s) started in SVC mode.
[......]

 

Job Done.

How To Configure BIND as a DNS Server

The aim is to setup two DNS servers:
Master DNS server – OpenWRT router -192.168.0.1 – FQDN: ns1.example.com
Slave DNS server – Debian server – 192.168.0.2 – FQDN: ns2.example.com
Web Server – 192.168.0.3 – FQDN: example.com

  1. Setting the Hostname on the Name Servers
  2. Install Bind on Both Name Servers
  3. Configure the Master Bind Server
  4. Configure the Slave Bind Server

1. Setting the hostname

Edit /etc/hosts file so it looks like this

127.0.0.1       localhost
192.168.0.1     ns1.example.com ns1

and for debian server:

127.0.0.1       localhost
192.168.0.2     ns2.example.com ns2

then edit the /etc/hostname so it looks like this:

ns1

and for debian box:

ns2

2. Instal BIND

On the OpenWRT router from the ssh run the following:
Uninstall preinstalled dnsmasq:

/etc/init.d/dnsmasq stop
opkg remove dnsmasq

then install BIND

opkg update
opkg install bind-server bind-tools

On a debian box run:

 sudo apt-get update
 sudo apt-get install bind9 bind9utils bind9-doc

3. Configure the Master Bind Server

On the OpenWRT box the bind directory structure is a bit different from the one on debian as there is only one file that holds all the config: /etc/bind/named.conf

We need to add few bits to it (highlighted in red)

// this section defines who is allow to submit queries to the server
acl goodclients {
        192.168.0.0/24;
        localhost;
        localnets;
};

options {
        directory "/tmp";

        recursion yes;
        allow-transfer { none; };

        dnssec-validation auto;

        auth-nxdomain no;    # conform to RFC1035
        listen-on-v6 { any; };
};

// prime the server with knowledge of the root servers
zone "." {
        type hint;
        file "/etc/bind/db.root";
};

// be authoritative for the localhost forward and reverse zones, and for
// broadcast zones as per RFC 1912

zone "localhost" {
        type master;
        file "/etc/bind/db.local";
};

zone "127.in-addr.arpa" {
        type master;
        file "/etc/bind/db.127";
};

zone "0.in-addr.arpa" {
        type master;
        file "/etc/bind/db.0";
};

zone "255.in-addr.arpa" {
        type master;
        file "/etc/bind/db.255";
};

// here we define our zones

zone "example.com" {
    type master;
    file "/etc/bind/zones/db.example.com";
    allow-transfer { 192.168.0.2; };
};

zone "0.168.192.in-addr.arpa" {
    type master;
    file "/etc/bind/zones/db.192.168.0";
    allow-transfer { 192.168.0.2; };
};

Create the Forward Zone File

In the /etc/bind folder create zones subfolder then create db.example.com file and paste this content:

$TTL 604800
@ IN SOA ns1.example.com. root.example.com. (
                        10 ; Serial
                    604800 ; Refresh
                     86400 ; Retry
                   2419200 ; Expire
                  604800 ) ; Negative Cache TTL
;

; Name servers
example.com. IN NS ns1.example.com.
example.com. IN NS ns2.example.com.

; A records for name servers
ns1 IN A 192.168.0.1
ns2 IN A 192.168.0.2

; Other A records
@         IN A 192.168.0.3
www       IN A 192.168.0.3
computer1 IN A 192.168.0.4
printer   IN A 192.168.0.5

IMPORTANT bit to remember here is to change the serial number each time this file is edited!

 

Create the Reverse Zone File

In /etc/bind/zones create file: db.192.168.0 and paste this content:

 

$TTL    604800
@       IN      SOA     example.com. admin.example.com. (
                 10       ; Serial
           604800         ; Refresh
            86400         ; Retry
          2419200         ; Expire
           604800 )       ; Negative Cache TTL
;

; Name servers
IN      NS      ns1.example.com.
IN      NS      ns2.example.com.

; PTR Records
1               IN      NS      ns1.example.com.
2               IN      NS      ns2.example.com.
3               IN      NS      www.example.com.
5               IN      NS      printer.example.com.
4               IN      NS      computer1.example.com.

Testing

named-checkconf /etc/bind/named.conf

If there is a problem with the config this will tell you where to look in the config file. If the config is fine there is no output.
Next we check our zones:

named-checkzone example.com /etc/bind/zones/db.example.com

If your file has no problems, it should tell you that it loaded the correct serial number and give the “OK” message;

zone example.com/IN: loaded serial 10
OK

then we do the same thing to the reverse lookup zone file.

if everthing is OK the we enable and start the BIND service:

/etc/init.d/named enable
/etc/init.d/named start

With the server running run this:

dig ANY intra @localhost

and if everything is setup correctly you should get something like this:

root@wrt:/etc/bind/zones# dig any example.com @localhost

; <<>> DiG 9.9.6-P1 <<>> any example.com @localhost
;; global options: +cmd
;; Got answer:
;; ->>HEADER<

 

Configure the Slave Bind Server

Configuring the Options File

Now on our debian box we edit the config file

nano /etc/bind/named.conf.options

Edit the options section so it looks like this:

options {
directory "/var/cache/bind";
recursion no;
allow-transfer { none; };

dnssec-validation auto;

auth-nxdomain no; # conform to RFC1035
listen-on-v6 { any; };
};

Save and close the file when you are finished.

Configuring the Local Configuration File

sudo nano /etc/bind/named.conf.local

We will create each of our zone specifications.

First, we will work on the forward zone:

zone "example.com" {
};

and edit it so it looks like this:

zone "example.com" {
type slave;
file "db.example.com";
masters { 192.168.0.1; };
};

This completes our forward zone setup.

We can use this same exact format to take care of our reverse zone config:

zone "0.168.192.in-addr.arpa" {
type slave;
file "db.192.168.0";
masters { 192.168.0.1; };
};

When you are finished, you can save and close the file.

To check that zone transfer was successful run:

sudo tail -f /var/log/syslog

That should have some entries to indicate that the zone files have been transferred correctly.

To configure DHCP follow here.

Good source of BIND config knowledge here.

Awstat on Debian with NGINX

AWStats is a free powerful and featureful tool that generates advanced web, streaming, ftp or mail server statistics, graphically. This log analyzer works as a CGI or from command line and shows you all possible information your log contains, in few graphical web pages.

Install

apt-get install awstats

On debian, awstats install things in 3 places :

    • /etc/awstats : contains config files
    • /usr/share/awstats : contains tools and libraries used by awstats
    • /usr/share/doc/awstats : contains docs, tools for building the static html pages, icons and other static files used by html

To get a list of files and locations on your system you can always run:

dpkg -L awstats

Overview

I have some custom needs, I have 3 domains :

  • domain1.com
  • domain2.com
  • main-domain.com

And I want to have stats for the first two domains. The main-domain.com is used as the master domain of the server, with awstats available at awstats.main-domain.com, instead of having domain1.com/awstats and domain1.com/awstats.

We also want to password protect the stats, but with different credential for each vhost.

These steps have been tested on Debian whezzy and jessie.

Formatting Nginx log

Nginx by default output logs that can be read by awstats:

error_log /path/to/log.log;

Config

The /etc/awstats folder should have:

  • awstats.conf.dist
  • awstats.conf.local

awstats.conf.dist is the main conf file, origin of all the other conf files. It’ll also fallbacks to this file if no other config file exists.

awstats.conf.local is an empty file. It’s the parent of all the other config files. This is where you put shared rules.

Next I copy the content of awstats.conf.dist into awstats.conf.local, and just put the important rules inside each vhost config, so they’re easier to read, and shorter. So we need to create two additional files;

touch awstats.domain1.com.conf

touch awstats.domain2.com.conf

in the empty config files we copy and edit this:

# Path to you nginx vhost log file
LogFile="/var/logs/access.log"

# Exclude local subnets and some public IPs that we do not want to include in the stats
SkipHosts="REGEX[^192\.168\.1\.] REGEX[^10\.10\.10\.] 11.11.11.11 22.22.22.33"

# Domain of your vhost
SiteDomain="domain1.com"

# Directory where to store the awstats data
DirData="/var/lib/awstats/"

# Other alias, basically other domain/subdomain that's the same as the domain above
HostAliases="domain1.com"

Fine tune the global settings

We need to edit the awstats.conf.local file, and:
1. Disable DNSLookup : DNSLookup = 0
2. Remove LogFile, SiteDomain, DirData and HostAliases directive, as they’re useless outside their context.
3. Set LogFormat to Combined: LogFormat = 1
4. Enable GeoIP (require additional steps-see below).

Configure GeoIP plugin

Install GeoIP to speed up the hostname lookups. This will significantly improve the performance since DNS lookups will generally take a long time. Using this tutorial, install the GeoIP library.

Next, download the latest GeoIP perl module -and- latest GeoLite country database.
http://www.maxmind.com/download/geoip/api/perl/ I usually run wget to download the file from shell: i.e.:

wget http://geolite.maxmind.com/download/geoip/api/perl/Geo-IP-1.36.tar.gz

or
download it from https://github.com/maxmind/geoip-api-perl

Extract the tar/zip file and inside the Geo-IP folder, run:

perl Makefile.PL
make
make test
make install

 

wget http://geolite.maxmind.com/download/geoip/api/c/GeoIP-1.4.5.tar.gz

Download and extract the file.  Inside the GeoIP 1.4.5 folder, run:

./configure
make
make check
make install

now add this bit to awstats config file:

LoadPlugin=”geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat”

Finalization

To calculate the stats, a perl script is available in /usr/share/doc/awstats/examples. The awstats_updateall.pl will compute the stats for each available config. It’s easy, just run :

/usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

Use cron job to run the script how often is necessary. If you logrotate make sure you run the script just before new log rotation.

Setting up nginx report website

This assumes that NGINX is already installed, configured and running OK. So we just need to create new vhost:

server {
    listen 80;
    server_name awstats.main-domain.com;
    root    /var/www/awstats;

    error_log /var/log/nginx/awstats.main-domain.com.error.log;
    access_log off;
    log_not_found off;

    location ^~ /icon {
        alias /usr/share/awstats/icon/;
    }

    location ~ ^/([a-z0-9-_\.]+)$ {
      return 301 $scheme://awstats.main-domain.com/cgi-bin/awstats.pl?config=$1;
}

        location ~ ^/cgi-bin/.*\\.(cgi|pl|py|rb) {
        gzip off;
        include         fastcgi_params;
        fastcgi_pass    127.0.0.1:9000;
        fastcgi_index   cgi-bin.php;
        fastcgi_param   SCRIPT_FILENAME    /etc/nginx/cgi-bin.php;
        fastcgi_param   SCRIPT_NAME        /cgi-bin/cgi-bin.php;
        fastcgi_param   X_SCRIPT_FILENAME  /usr/lib$fastcgi_script_name;
        fastcgi_param   X_SCRIPT_NAME      $fastcgi_script_name;
        fastcgi_param   REMOTE_USER        $remote_user;
    }
}

Create the /etc/nginx/cgi-bin.php file

 array("pipe", "r"),  // stdin is a pipe that the child will read from
    1 => array("pipe", "w"),  // stdout is a pipe that the child will write to
    2 => array("pipe", "w")   // stderr is a file to write to
);

$newenv = $_SERVER;
$newenv["SCRIPT_FILENAME"] = $_SERVER["X_SCRIPT_FILENAME"];
$newenv["SCRIPT_NAME"] = $_SERVER["X_SCRIPT_NAME"];

if (is_executable($_SERVER["X_SCRIPT_FILENAME"])) {
    $process = proc_open($_SERVER["X_SCRIPT_FILENAME"], $descriptorspec, $pipes, NULL, $newenv);
    if (is_resource($process)) {
        fclose($pipes[0]);
        $head = fgets($pipes[1]);
        while (strcmp($head, "\\n")) {
            header($head);
            $head = fgets($pipes[1]);
        }
        fpassthru($pipes[1]);
        fclose($pipes[1]);
        fclose($pipes[2]);
        $return_value = proc_close($process);
    } else {
        header("Status: 500 Internal Server Error");
        echo("Internal Server Error");
    }
} else {
    header("Status: 404 Page Not Found");
    echo("Page Not Found");
}
?>

Now I copy the cgi-bin folder from relevant awstats location to my awstats vhost folder: /var/www/awstats and also do not forget to apply the correct permissions.

Now if everything is setup correctly you should be able to access your stats at:
http://awstats.main-domain.com/domain1.com
http://awstats.main-domain.com/domain2.com

if for some reason the above links do not work try:
http://awstats.master-domain.com/cgi-bin/awstats.pl?config=domain1.com

 

Pi vs UDOO MySQL performance test

Create test DB:

mysql -u root -p
mysql> create database test;
exit;

Prepare the database:

sysbench --test=oltp --oltp-table-size=100000 --mysql-db=test --mysql-user=root --mysql-password=mysqlpassword prepare

Run the test:

sysbench --test=oltp --oltp-table-size=100000 --mysql-db=test --mysql-user=root --mysql-password=mysqlpassword --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run

Pi Output:

OLTP test statistics:
    queries performed:
        read:                            28364
        write:                           0
        other:                           4052
        total:                           32416
    transactions:                        2026   (33.76 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 28364  (472.67 per sec.)
    other operations:                    4052   (67.52 per sec.)

Test execution summary:
    total time:                          60.0080s
    total number of events:              2026
    total time taken by event execution: 59.8767
    per-request statistics:
         min:                                 12.90ms
         avg:                                 29.55ms
         max:                                 76.67ms
         approx.  95 percentile:              50.34ms

Threads fairness:
    events (avg/stddev):           2026.0000/0.00
    execution time (avg/stddev):   59.8767/0.00

UDOO output:

OLTP test statistics:
    queries performed:
        read:                            287532
        write:                           0
        other:                           41076
        total:                           328608
    transactions:                        20538  (342.19 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 287532 (4790.71 per sec.)
    other operations:                    41076  (684.39 per sec.)

Test execution summary:
    total time:                          60.0186s
    total number of events:              20538
    total time taken by event execution: 479.3667
    per-request statistics:
         min:                                  7.26ms
         avg:                                 23.34ms
         max:                                196.47ms
         approx.  95 percentile:              32.59ms

Threads fairness:
    events (avg/stddev):           2567.2500/50.73
    execution time (avg/stddev):   59.9208/0.01

the important bit is the transactions line:
UDOO: transactions: 20538 (342.19 per sec.)
PI: transactions: 2026 (33.76 per sec.)

Last step – drop the test database:

mysql -u root -p
mysql> drop database test;
exit;