Compiling RetroShare for the Raspberry Pi, revisited

With my emulated Raspberry Pi set up, I wanted to compile the newest version of RetroShare to check both whether the pseudo-cross-compilation actually works and if my last how-to is still valid.

At the time of writing this, the RetroShare wiki gives these instructions:

sudo apt-get install -y g++ libbz2-dev libcunit1-dev libgnome-keyring-dev libgpg-error-dev libgpgme11-dev libprotobuf-dev libqt4-dev libssh-dev libssl-dev libupnp-dev libxss-dev qt4-qmake subversion

cd ~/

svn co svn://svn.code.sf.net/p/retroshare/code/trunk retroshare

cd ~/retroshare/libbitdht/src && qmake && make clean && make && \

cd ~/retroshare/openpgpsdk/src && qmake && make clean && make && \

cd ~/retroshare/libretroshare/src && qmake && make clean && \

cp ~/retroshare/libretroshare/src/Makefile ~/retroshare/libretroshare/src/Makefile.old &&\

cat ~/retroshare/libretroshare/src/Makefile | perl -pe 's/^(INCPATH[^\n].*)\n/$1 -I\/usr\/lib\/arm-linux-gnueabihf\/glib-2.0\/include\n/g'  > ~/retroshare/libretroshare/src/Makefile.new &&\

mv ~/retroshare/libretroshare/src/Makefile.new ~/retroshare/libretroshare/src/Makefile &&\

make && \ 

cd ~/retroshare/retroshare-nogui/src && qmake && make clean && make && \

cd ~/retroshare/retroshare-gui/src && qmake && make clean && make

I want to play it safe and not use the latest version from svn, because the project is very active at the moment and I don't want to sit here and wonder whether the eventual compiler errors are my fault or a result of an incomplete commit there.

So, download the latest version: http://sourceforge.net/projects/retroshare/files/RetroShare/0.5.4d/RetroShare-v0.5.4d.tar.gz

Problems with architectural chroot

First of all, it seems like the chroot isn't perfect. For instance, I can get an internet connection, so wget fails, and I also can't sudo.

As a consequence of that, the installation of packages has to be done from within the QEMU environment. A lot has changed since version 0.5.4b, so I'll stick to what their wiki says about what we need to install.

sudo apt-get install -y g++ libbz2-dev libcunit1-dev libgnome-keyring-dev libgpg-error-dev libgpgme11-dev libprotobuf-dev libqt4-dev libssh-dev libssl-dev libupnp-dev libxss-dev qt4-qmake

I was about to say "Activate the swapfile", but we don't need it, because we have the RAM of our host linux machine.

Exit the QEMU environment now with sudo reboot.

Compiling RetroShare

Enter the architectural chroot like described in my last post.

Then, enter the development directory and start compiling. I've adapted the above given commands to fit my directory structure and increase the number of threads to make use of my i5's four cores.

cd ~/development/RetroShare-v0.5.4b/trunk/

cd libbitdht/src && qmake && make clean && make -j4 && \

cd ../../openpgpsdk/src && qmake && make clean && make -j4 && \

cd ../../libretroshare/src && qmake && make clean && \

cp Makefile Makefile.old &&\

cat Makefile | perl -pe 's/^(INCPATH[^\n].*)\n/$1 -I\/usr\/lib\/arm-linux-gnueabihf\/glib-2.0\/include\n/g'  > Makefile.new &&\

mv Makefile.new Makefile &&\

make -j4 && \ 

cd ../../retroshare-nogui/src && qmake && make clean && make -j4 && \

cd ../../retroshare-gui/src && qmake && make clean && make -j4

Wow! That was incredibly fast! It worked like a charm and it only took 30 minutes. If this isn't a full-scale success, I don't know what is ;)

The only thing left to do now is to strip the executable

strip RetroShare

and copy it to the actual Raspberry Pi. It works like a charm!

A huge "Thank you" to the guys from RetroShare for making it easier and easier to compile this awesome piece of software for the Raspberry Pi!

Faster compiling on an emulated Raspberry Pi on Linux

In my last article about RetroShare on the Raspberry Pi, I've written about my experiences ...

Unfortunately, the 256 MB RAM of my Raspi is barely enough to keep RetroShare running, but even a simple 'sudo apt-get update' can crash it. The RetroShare project itself is very active at the moment, bugs are constantly fixed and features added. But compiling RetroShare requires all resources, so I thought about alternatives.

"Simple", I thought, "Emulation!"

Spoiler alert: Emulating a Raspberry Pi using QEMU is slow.

I have no experience with emulation whatsoever, so there is no real reason for choosing QEMU over other emulators like VirtualBox, I just found instructions and information for this way first.

What we need to get started

The Emulator

My desktop PC is running Ubuntu 12.04, the rest of the system specifications shouldn't matter. (Yes, I know that this version is a bit outdated, but as a True Believer in Murphy's Law I tend to not change a running system.)

Installing QEMU

Install the package qemu-kvm-extras the way you usually install packages, e.g., using aptitude or

sudo apt-get install qemu-kvm-extras

This will also install some dependencies that are needed.

You will also need to download a QEMU-ready linux kernel for the Raspberry Pi, which you can do here: http://xecdesign.com/downloads/linux-qemu/kernel-qemu
Alternatively, you can compile your own kernel.

Preparing the environment

Create a directory in which our experiment will take place. I chose:

mkdir ~/development/raspberrypi-qemu

Raspbian Wheezy

I used newest version of Raspbian Wheezy currently available at http://www.raspberrypi.org/downloads:

Torrent	2012-12-16-wheezy-raspbian.zip.torrent
Direct download	2012-12-16-wheezy-raspbian.zip
SHA-1	514974a5fcbbbea02151d79a715741c2159d4b0a
Default login	Username: pi Password: raspberry

Download it and unpack the image file into the directory you prepared in the last step.

Plan A: Running an emulated Raspberry Pi

In the directory of the image, run the following command:

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1" -hda 2012-10-28-wheezy-raspbian.img

The parameters have the following functions:

-kernel kernel-qemu

the QEMU-ready kernel we just downloaded

-cpu arm1176

the CPU we want to emulate (ARM1176JZF-S (700 MHz))

-m 256

how much RAM should be available (in MB)

-M versatilepb

the machine type to emulate

-no-reboot

exit instead of rebooting

-serial stdio

redirects the serial port to the standard I/O

-append "root=/dev/sda2 panic=1"

where to find the root partition, depends on the image

-hda 2012-10-28-wheezy-raspbian.img

what should be used as hard drive, in this case our image

Now, you might be tempted to simply increase the amount of available RAM, but it doesn't work that way. I'm not sure why, but it only works with 256 MB RAM. On the other hand, we can circumvent the problems of creating a swap file on an SD card, because most likely the image is not located on one. Increasing the available memory is easy now, just add a sufficiently large swap file.

We also shouldn't forget to resize our image, because right now we only have about 200 MB of free space left. Again: There already are many articles on the net covering this, so I'll only quickly describe what I did.

1. With QEMU not running, you can use qemu-img to resize an image:
   

   qemu-img resize 2012-12-16-wheezy-raspbian.img +1G

2. For other reasons, raspian's built-in functionality of growing the partition to fill the SD card won't work here, so I did it the hard way.
3. Boot your emulated Raspberry Pi again using QEMU
4. Resize the partition using fdisk
   

   sudo fdisk /dev/sda

   It should look similar to this:
   

   Device Boot  Start       End    Blocks   Id  System
   /dev/sda1     8192    122879     57344    c  W95 FAT32 (LBA)
   /dev/sda2   122880   5885951   2881536   83  Linux

   You need to delete partition 2 and create it again with the same start, but this time with the highest allowed value for end.
5. Resize the filesystem using
   

   sudo resize2fs /dev/sda2

It's too slow

Before we start compiling RetroShare again, let's check up on the speed. This is what I get from the QEMU Raspberry Pi:

pi@raspberrypi:~$ cat /proc/cpuinfo | grep MIPS
BogoMIPS        : 565.24

And this is from my actual Raspberry Pi

pi@raspberrypi:~$ cat /proc/cpuinfo | grep MIPS
BogoMIPS        : 697.95

So, my emulated raspi running on a Intel Core i5 is actually slower than the real raspi ... which is not what I wanted. I mean, okay, if I'd have the machine running for something else anyway, that wouldn't be a problem. But I still want it to be faster.

Plan B: Architectural chroot a.k.a. chroot Voodoo

Looking for solutions to speed up QEMU, I stumbled upon another approach: architectural chroot!

Now, I'm familiar with chroot (at least I though so) and I've used it hundreds of times when my Ubuntu got f*cked up because of an update or some other stuff. And I remember the difficulties when I tried to chroot into a 64 bit system from a 32 bit Live-CD. But it seems like there is a way around this. Coincidentally, we're already close to what we need: a static version of QEMU.

Install it via apt-get (or build it yourself)

sudo apt-get install qemu-user-static

We need to mount the image using loopback, but since the image contains multiple partitions, we require kpartx

$ sudo kpartx -a -v 2012-12-16-wheezy-raspbian.img 
add map loop0p1 (252:8): 0 114688 linear /dev/loop0 8192
add map loop0p2 (252:9): 0 5763072 linear /dev/loop0 122880

$ sudo mount /dev/mapper/loop0p2 /mnt/temp

Now, copy the static QEMU binary TODO and mount the special directories:

sudo cp /usr/bin/qemu-arm-static /mnt/temp/usr/bin
sudo mount -o bind /dev /mnt/temp/dev
sudo mount -o bind /proc /mnt/temp/proc
sudo mount -o bind /sys /mnt/temp/sys

Before we can enter the chroot environment, it's time for the magic! As root (simple sudo won't work), do this (all in one line):

echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:' > /proc/sys/fs/binfmt_misc/register

This registers the static QEMU we copied as arm-interpreter to the kernel. The path specified needs to be the same on both your linux machine and the Raspberry Pi environment.

Now we can chroot:

sudo chroot /mnt/temp

Did it work?

$ uname -a
Linux localhost 2.6.32 #58-Ubuntu SMP Thu Jan 24 15:28:10 UTC 2013 armv7l GNU/Linux

Hooray! Welcome to your (much faster) Raspberry Pi environment! Now, let's do some compiling ;)

Cleaning up

To avoid inconsistencies, make sure you never use QEMU and chroot at the same time! Even more, you need to completely unmount the image before you start QEMU. Otherwise you might see some undesireable side effects.

sudo umount /mnt/temp/dev
sudo umount /mnt/temp/proc
sudo umount /mnt/temp/sys
sudo umount /mnt/temp

sudo kpartx -d -v 2012-12-16-wheezy-raspbian.img

Acknowledgements

I also want to give credit to the following articles that helped me doing this:

• http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/
• http://www.smallbulb.net/2012/225-emulating-raspberry-pi
• http://xecdesign.com/compiling-a-kernel/
• http://www.frambozenbier.org/index.php/raspi-distribution-news/11474-jojopi-on-fedora-arm-on-raspberry-pi
• http://www.darrinhodges.com/chroot-voodoo/

Compiling RetroShare for the Raspberry Pi

The question is not if you are paranoid,
it is if you are paranoid enough.

I'm a fan of darknets. I've always been. I don't want this to turn into a political discussion - it is a "tech" blog - so it should be sufficient to say that I like and use RetroShare.

RetroShare is a decentralized sharing application/network that is, in my opinion, the best darknet software available right now. I've already tried WASTE, GnuNet and AllianceP2P, and they couldn't convince me for long.
Maybe I'll make an article about the pros and cons of RetroShare, but in this article I want to describe my attempts to get it running on my Raspberry Pi, so it can serve as a 24/7 node in the RetroShare network to forward connections without consuming that much power.

When I heard about the Raspberry Pi, the geek inside me wanted to have one. Fabian is still pissed that my Raspi arrived sooner than his, that is it arrived at all, although I ordered it much later. (I think he wants to go for Parallella now ...)

I've already managed to compile RetroShare 0.5.3 on and for my Raspi, but it segfaulted whenever I wanted to add a friend or open the settings menu. With the recent release of 0.5.4 I thought I could give it a try again.

Be warned: The compiling alone takes a few hours. I didn't write down exact times, but it's probably best if you have some other tasks to attend to for 2 hours while one subproject is compiling.

What we need to get started

Raspberry Pi and Raspbian Wheezy
I'm starting with my Raspberry Pi (Model B) with a sufficiently large SD card (4 GB should be enough) and a freshly installed Raspbian Wheezy (build 2012-09-18), which I've updated today (15th October) doing apt-get update && apt-get upgrade, and haven't modified so far besides putting my dotfiles under revision control using git.

In case that information changes on the Raspi website, I'll duplicate the links and information here, so we're all on the same page:

Torrent	2012-09-18-wheezy-raspbian.zip.torrent
Direct download	2012-09-18-wheezy-raspbian.zip
SHA-1	3bc788d447bc88feaae8382d61364eaba1088e78
Default login	Username: pi Password: raspberry

RetroShare sources
The RetroShare version we need is 0.5.4b

Create a directory development in your home directory, cd there, download the sources and unpack them.

mkdir ~/development
mkdir ~/development/RetroShare-v0.5.4b
cd ~/development
wget http://sourceforge.net/projects/retroshare/files/RetroShare/0.5.4b/RetroShare-v0.5.4b.tar.gz
cd RetroShare-v0.5.4b
tar -xvf ../RetroShare-v0.5.4b.tar.gz

Required packages

My primary source for instructions is the RetroShare wiki page UnixCompile of today's version. It is a bit outdated because they've removed gnupg and introduced openpgp, but didn't update their instructions. Install the following packages with:

sudo apt-get install libqt4-dev g++ libupnp-dev libssl-dev libgnome-keyring-dev libbz2-dev libxss-dev

Then you need to compile the projects in the subdirectories, but there are still some modifications necessary to make it work under Debian. The first two subprojects should work fine:

cd ~/development/RetroShare-v0.5.4b/trunk/
cd libbitdht/src && qmake && make clean && make -j2
cd ../../openpgpsdk/src && qmake && make clean && make -j2

The problems start with libretroshare and there are two changes you need to make on it:

Firstly, because even though the preprocessor commands are there, you still need to tell the make process that you are on Debian and your libupnp is of a different version. You do this by editing the file libretroshare.pro in the libretroshare/src/ directory and after the line 221

DEFINES *= UBUNTU

you add (not replace) these lines

DEFINES *= DEBIAN
DEFINES *= UPNP_VERSION=10617

because we do in fact have version 1.6.17 of libupnp installed. (See this thread in the RetroShare forum.)

Secondly, someone hardcoded the location of glib-2.0 into that project file in a very system dependent way. You need to change this line (should be directly below the previous change, line 224 now)

INCLUDEPATH += /usr/include/glib-2.0/ /usr/lib/glib-2.0/include

to this

INCLUDEPATH += $$system(pkg-config --cflags glib-2.0 | sed -e "s/-I//g")

(Source)

cd ../../libretroshare/src && qmake && make clean && make -j2

And now the real fun starts. The Raspberry Pi has 256 MB RAM of which at least 16 MB need to be reserved for the video core, so only 240 MB RAM left. Unfortunately, this is not enough to compile retroshare-gui, it will quit with something like

virtual memory exhausted: Cannot allocate memory
g++: internal compiler error: Killed (program cc1plus)

when it tries to compile qrc_images.cpp.

Super hot update
The irony of situation is that just today an enhanced version of the Raspberry Pi was announced with 512 MB of RAM, which should make the following swap file part obsolete.

Now, I've heard that very, very bad things happen if you create a swap file on a flash storage device like the SD card your Raspi uses or USB sticks. But as a matter of fact, Raspbian wheezy already uses a swap file per default, so it can't be that bad. All you need to do is create another swap file of sufficient size, say 256 MB, which you delete afterwards just to be on the safe side:

sudo dd if=/dev/zero of=swapfile bs=1M count=256
sudo mkswap swapfile
sudo swapon swapfile

Then you can compile retroshare-gui

cd ../../retroshare-gui/src && qmake && make clean && make -j2

and deactivate the swap file again:

sudo swapoff swapfile
sudo rm swapfile

This is also a good opportunity to turn of the default swap file, too, as it really isn't that healthy for your SD card. (Hoping that actually running RetroShare is possible with only 256 MB RAM)

Done

And that's it, you've just compiled RetroShare. This article is already way too long, so I'll stop here and post about configuring and running it in a later post. Hopefully soon.