[Distro] Raspberry Server: Web Edition

[Coburn]

Excuse me if this is in the wrong section, but here goes nothin'.

Raspberry Server: Web Edition
Built by Coburn's Domain.

What's this?
Raspberry Server: Web Edition is a linux distro that has been optimized for use as a web server - if you ever wanted to run your own wiki, blog or anything on the R-Pi, this is for you. Many tweaks have been applied and things have been stripped from the official Raspbian Wheezy base for a lean, mean, fighting web serving machine.

It features the following:

• Really fast boot time - less than 45 seconds, unlike 1min+ for the official build
• Official Linux Kernel (if memory serves me correctly, 3.2.x from the Raspbian build) with mini-turbo (800Mhz OverClock using CPUFreq ondemand governor - originally I wanted to use 1GhzNitroBoost(TM) but I thought that not all R-Pis are made equal and I didn't want to have burnt Pis' in my mailbox)
• Lighttpd Web Server - Apache was too performance crippling.
• PHP 5.4.x in Fast-CGI configuration for high performance
• MySQL Server 5.5.x with InnoDB disabled (since it's a memory consuming thing)
• PHPMyAdmin
• Ruby, SQLite 3, Perl, Python 2.7 (minimal), Python 3 & 3.2 (minimal)
• PHP Extension Suite, with all the goodies (gd, imagemagick, mysql, mcrypt, etc)
• PureFTPd as FTP Server
• OpenSSH Server
• High performance Network I/O Tweaks
• Custom daemons for memory cleaning and network time sync since the R-Pi doesn't have a realtime clock.
• -- Fake-HWClock serves as a "fake" clock so the R-Pi doesn't go back to 1-1-1970 12:00AM

As mentioned, a lot of bloat has been removed and additionally, the GPU is disabled and Xorg is not installed. A server doesn't need Xorg, you can just use the HDMI port with 'htop' or 'top' running and use that as a monitoring console.

Initial testing has shown that the R-Pi is quite snappy and doesn't seem get flogged to much when running a few tests using this Lighttpd and PHP in FastCGI setup. Unlike traditional FastCGIs done with lighttpd, Raspberry Server uses a better approach that is more robust and speedy.

Of course, speed will vary on the SD card Read & Write speeds and class. All SD Cards that come pre-loaded with this distro will be class 6 or higher and will be available in the Coburn's Domain Store.

How do I get it?
Raspberry Server: Web Edition will be made available in 2 flavours, do it yourself - via a Disk Image that you write to your SD Card yourself (DIY) or pre-made ready-to-run SD Card - available for purchase from the Coburn's Domain Store. (Yes, I do ship internationally).

Questions, comments and observations are more than welcome!
The R-Pi platform is great, and I hope to make future versions of this distro plus more for various other uses.

Cheers!

[neophytte]

I'm keen, and in Australia - PM me the details

http://neophytte.mine.nu/

Cheers

Richard

[dipole]

Awesome! Looking forward to this. WIll you provide us with instructions on how to roll our own?

[sjfaustino]

Good work, always nice to have a specific platform to build on.
Only sugestion that I would do is to replace Apache with lighttpd or nginx.

[Coburn]

Awesome! Looking forward to this. WIll you provide us with instructions on how to roll our own?

I will eventually. I won't want to let the cat out of the bag just yet!

Good work, always nice to have a specific platform to build on.
Only sugestion that I would do is to replace Apache with lighttpd or nginx.

I will do that in a later update of Raspberry Server Web Edition. The reason Apache came installed is because 95% of the sites on the net use it and it's for no-brainers. Wordpress works fine on it and all that jazz - and that is what appeals to users - they can slap up a Wordpress install and know it's gonna work, without issues.

[big250]

like it!!!!!!!!!

[tomwilson]

Great idea!

[daveg]

I'll be keen to try it out when available

[poing]

Interesting, although I found phpAdmin to be incredibly slow on the Pi. But then maybe I didn't configure it well or something.

[pigeorgetaylor]

Brilliant idea! I cant seem to get my server running on arch... But i also found PHPmyadmin to be slow when I had my raspbian server running. Raspbian was too bloated for a server, although X graphical interface was useful at times.
Got an ETA

[Coburn]

Private Test Build 1.0a is coming out of the oven. I have removed Apache for Lighttpd, mainly because I noticed Apache can bog the Raspberry Pi down at times. It also brings speed improvements.

ETA should be within 2 weeks.

[milhouse]

I found the following three commands (executed as root, eg. "sudo su - " beforehand) are sufficient to remove most if not all the X11/desktop (and associated) packages. That is, turn regular Raspbian Wheezy into a minimalist "server" version:

Code: Select all

# Remove libx11-6 to remove x11 and all dependent packages (anything GUI related, basically)
apt-get remove libx11-6 libgtk-3-common xkb-data lxde-icon-theme raspberrypi-artwork penguinspuzzle

# Clean up redundant packages
apt-get autoremove

# Purge config files for removed packages
dpkg -l | awk '/^rc/ {print $2}' | xargs dpkg --purge

The above removes several hundred megabytes of packages you don't need when running a headless "server" - should make apt-get upgrade that bit quicker as you are no longer updating/maintaining stuff you don't use.

[Coburn]

One security flaw that I spotted is that the pi user can run anything with sudo. This is dangerous in the server environment.

I have re-enabled sudo password prompting as it would give the user a layer of protection, hopefully.

I also have taken milhouse's commands into account and was unable to remove some packages since PHP would be crippled if I did so. However, that penguin puzzle game has hit the trash.

I will also like to say that once if you're going to take up the plug 'n' play SD Card offer, I'll be selling the configuration in 8GB ("Lo-Fat") and 16GB ("Full-Fat") editions. This will just give you more storage for your server.

[Coburn]

Oh, and just for the record... you jelly?

Code: Select all

pi@raspsrv-test ~ $ free
                  total         used        free  shared  buffers   cached
Mem:        237872 68564 169308           0      2092    44464
-/+ buffers/cache:   22008  215864

[milhouse]

I also have taken milhouse's commands into account and was unable to remove some packages since PHP would be crippled if I did so. However, that penguin puzzle game has hit the trash.

In a freshly minted Raspbian there is no PHP to be removed, so I'm guessing you're running the above commands *after* installing other packages, and if those packages are left in a broken state I can only imagine thats because those packages have incorrect dependencies (depending on X11 without declaring so).

To prepare Raspbian for "server" operation (sans X11/desktop/etc.), it's best to run the above commands *before* installing any additional packages.

Or does PHP fail to install once X11/etc. has been removed? That again would sound like a suspect dependency...

My usual way of preparing Raspbian for my own personal "server" environment is:

1. Image Raspbian Wheezy to SD card
2. Boot Raspbian, complete raspi-config (timezone, expand rootfs etc.)
3. Run a setup script which does the following while I make a cup of tea...
   1. apt-get update
   2. Remove X11/desktop packages (see above commands)
   3. Install and configure my "server" packages (Logitech Media Server, Sickbeard, Transmission, dnsmasq, MySQL, NUT, backup cron job, etc.)
   4. apt-get update
   5. apt-get upgrade

Takes about 15 minutes start to finish.

[milhouse]

A useful command that will list installed packages sorted by their installed package size:

Code: Select all

dpkg-query -W --showformat='${Installed-Size;-10}\t${Section;-15}\t${Package;-30}\n' | sort -g

This can be helpful when looking for packages to remove to recover space, or that are simply no longer required (in a server).

[Coburn]

I also have taken milhouse's commands into account and was unable to remove some packages since PHP would be crippled if I did so. However, that penguin puzzle game has hit the trash.

In a freshly minted Raspbian there is no PHP to be removed, so I'm guessing you're running the above commands *after* installing other packages, and if those packages are left in a broken state I can only imagine thats because those packages have incorrect dependencies (depending on X11 without declaring so).

Well, yeah, it wanted some X11 libs for ImageMagick and few other tidbits.

Also, is it safe if I toss "raspberrypi-bootloader" ? It seems to be chewing up 89MB of disk space...

[milhouse]

Also, is it safe if I toss "raspberrypi-bootloader" ? It seems to be chewing up 89MB of disk space...

I think that package contains the firmware and kernel, so if you remove it you may not be able to upgrade to updated firmware and kernel (assuming you're actually left with a bootable system after removing it!)

[Coburn]

Alright, I think I'll not uninstall that...

Just doing a few more bits and pieces. Once I can confirm it works fine, I might churn out a public test build for you guys to test for me. Once I get feedback, I'll let it go live.

[Coburn]

Just a question:

What log files do I nuke when cleaning up? I have "echo > file.ext" the following files, shown as 1 byte:

Code: Select all

total 172
drwxr-xr-x 11 root       root       4096 Oct 17 07:18 .
drwxr-xr-x 12 root       root       4096 Oct 15 15:23 ..
-rw-r--r--  1 root       root          1 Oct 17 07:17 alternatives.log
drwxr-xr-x  2 root       root       4096 Sep 18 21:51 apt
-rw-r--r--  1 root       root      14049 Oct  2 11:51 aptitude
-rw-r-----  1 root       adm           1 Oct 17 07:17 auth.log
-rw-r--r--  1 root       root          1 Oct 17 07:17 bootstrap.log
-rw-rw----  1 root       utmp       1920 Oct  2 12:12 btmp
drwxr-xr-x  2 root       root       4096 Sep 18 22:39 ConsoleKit
-rw-r-----  1 root       adm           1 Oct 17 07:17 daemon.log
drwxr-xr-x  2 root       root       4096 Oct  2 12:09 dbconfig-common
-rw-r-----  1 root       adm        2224 Oct 16 09:54 debug
-rw-r-----  1 root       adm       11431 Oct 16 09:54 dmesg
-rw-r-----  1 root       adm       11431 Oct 15 17:35 dmesg.0
-rw-r--r--  1 root       root          1 Oct 17 07:17 dpkg.log
-rw-r--r--  1 root       root      24024 Oct  2 11:55 faillog
-rw-r--r--  1 root       root          1 Oct 17 07:17 fontconfig.log
drwxr-xr-x  2 root       root       4096 Sep 18 21:30 fsck
-rw-r-----  1 root       adm           1 Oct 17 07:17 kern.log
-rw-rw-r--  1 root       utmp     292292 Oct 16 11:55 lastlog
drwxr-x---  2 www-data   www-data   4096 Oct 15 15:23 lighttpd
-rw-r-----  1 root       adm           1 Oct 17 07:17 lpr.log
-rw-r-----  1 root       adm           0 Sep 18 22:39 mail.err
-rw-r-----  1 root       adm           0 Sep 18 22:39 mail.info
-rw-r-----  1 root       adm           1 Oct 17 07:17 mail.log
-rw-r-----  1 root       adm           0 Sep 18 22:39 mail.warn
-rw-r-----  1 root       adm           1 Oct 17 07:18 messages
drwxr-s---  2 messagebus adm        4096 Oct  2 11:56 mysql
-rw-r-----  1 messagebus adm           0 Oct 15 17:09 mysql.err
-rw-r-----  1 messagebus adm           1 Oct 17 07:17 mysql.log
drwxr-xr-x  2 root       root       4096 Sep 18 22:39 news
-rw-------  1 root       root          1 Oct 17 07:17 php5-fpm.log
drwxr-xr-x  2 root       root       4096 Oct 16 09:59 pure-ftpd
-rw-r--r--  1 root       root          1 Oct 17 07:17 regen_ssh_keys.log
drwxr-x---  2 root       adm        4096 Aug  6 01:02 samba
-rw-r-----  1 root       adm         387 Oct 16 16:39 syslog
-rw-r-----  1 root       adm           1 Oct 17 07:17 user.log

The reason I'm asking is that I want to wipe out old logs since they would be related to my R-Pi (especially dmesg) and I don't think we want people to be confused.

[milhouse]

I'd be tempted to nuke them all prior to shutdown, or mount the partition in another OS and remove them (this latter method would guarantee you had no log files relating to yourself). In theory log files should be recreated automatically on first boot.

If there are any problems with processes that can't create their own log file, I'd suggest creating empty files with the same ownership/file permissions.

[davros]

just what i had in mind...... cant wait to try this out

[tomgravell]

Looking forward to this!

[benzeman]

Subbed to the thread. This looks awesome - have you had a chance to test performance on a 512MB version? I've got one due today or tomorrow from Farnell, so I'm happy to test - I can give you an FTP server to upload the image to if you want?

I might also be able to help with hosting - PM me if you're interested!

Anyway, good luck!

[Coburn]

Subbed to the thread. This looks awesome - have you had a chance to test performance on a 512MB version? I've got one due today or tomorrow from Farnell, so I'm happy to test - I can give you an FTP server to upload the image to if you want?

I might also be able to help with hosting - PM me if you're interested!

Anyway, good luck!

I don't have a 512MB R-Pi to test it out on - I only have my trusty 256MB model. If someone has the 512MB model, could they supply me the bootloader files (and the kernel, if it's different?) so I can do a 512MB-version - at least a replacement bootloader partition tarball.

What I do hope to is that I can sell a few SDs with this distro preloaded, so I can then buy a 512MB model.

Hosting would be great, as I do have Japan and US mirrors that are used for my websites (Coburn's Domain, and Dandere Networks, the server network I own). However, both mirrors do randomly get pelted with bandwidth at times (and they are metered - 200GB each), so if I could get something like a round-robin checker on 3rd party hosts that I authorise (maybe even a cronjob) that queries a special URL and if the server replies there's a new build out, it would D/L it and all that jazz.

It's getting dark here and I need to get away from the office, so tomorrow I will upload the public test version. However, I request this version to not be redistrubuted (I know someone would break my request) since it's a test release and I don't want to type a zillion one-line replies saying "Update your installation", "Update", "Just update!", "UPDATE YOUR SD INSTALL!" etc.

This distro will require some love with fdisk, hdparm, tar and mountpoints. If you aren't that savvy with linux - wait until I have pre-made SD Cards and buy one of them for zero-fuss installation.

Otherwise I will help you to the best I can - I am not going to be held reliable if your Raspberry Pi decides it's too awesome and explodes into fireworks, this software makes your girlfriend leave you, you get a catroast (cat pees on it and BZZT) among other things.

In fact, you will need to use Linux to install this distro. Better download a live CD of Fedora or Ubuntu and get to know how to use it!