DonCharisma.org, don charisma, Seagate GoFlex Net

Build your own Pro NAS – Seagate Goflex Net with Debian Linux, RAID1 and OpenMediaVault


The result of spending copious hours and days, knee deep in Linux IS – a working and stable 2TB NAS with Debian Linux based on Seagate GoFlex NET. Persistence and Charisma eventually paid off, ended up being a labour of love, but it was WORTH IT. This is a guide on how to do the same the EASY WAY.

DonCharisma.org, don charisma, Seagate GoFlex Net
DonCharisma.org Seagate GoFlex Net

This is a fairly hardcore and long geeky guide (6000 words), I realise it’s a niche audience that will be interested in hacking cheap bits of hardware off of Amazon, so if not for you then don’t read it:)

WHY?

I like saving money and being green at the same time is a bonus.

Seagate GoFlex Net has two SATA ports for portable hard drives, it has a 1.2GHz processor and runs on just a few watts of electricity. For it’s size it’s powerful, expandable and green. Free market prices vary obviously, at the time of writing it’s not expensive around £29 in UK, I guess that’s $40 in USA. Total of around £130 for a two disk RAID1 NAS with a web based GUI for administration, off the shelf alternatives are maybe £200-£250. Ability to install any of the 25000 debian packages, priceless.

Seagate package this device with built in software from Pogoplug. Seagate in my experience are THE BEST hard drive manufacturer. However they tend to suck big style on the software they include with their hardware. Pogoplug on GoFlex Net is like putting a lawnmower engine in a Ferrari. It’s OK for some consumers who don’t want to know anything about the technical side, but really it’s not much use for anyone serious. Also it poses some security issues, such as sharing data on Windows network without any password, meaning anyone who has access to your local network has access to your shared files.

I wanted a semi-portable NAS. Seagate GoFlex Net weighs only a few hundred grammes, hand luggage for travelling.

I didn’t want to mess around too much with circuit boards, a finished device in a nice case fits the bill. GoFlex Net has a nice moulded plastic case.

I wanted the device to be a NAS, that is accessible on the Network so I don’t have to mess around plugging and unplugging USB cables. GoFlex Net has gigabit ethernet port.

As I don’t want to spend all my time knee deep in Linux commands, I wanted some kind of easy-to-use management interface. OpenMediaVault fits the bill, everything in one place.

I wanted RAID1 for integrity of my data. GoFlex Net has 2 SATA ports, so RAID 0 and 1 and also spanning possible.

HOW?

Looking around on the internet, I looked at various different devices. Perhaps a Raspberry PI, but doesn’t have case or SATA interface for drives. Pogoplug? expensive and cheaper models don’t have SATA.

Seagate GoFlex Net was the best I could find on a miser’s budget and having the possibility of doing what I want it to do. I like the way the hard drives hot plug into the top of the device. Just a great design.

Stock software on GoFlex Net is Pogoplug, basically it’s Pogoplug with a Seagate label on it. Searching internet revealed that other’s had installed Arch Linux and Debian Linux on the device, and personally I have a preference for Debian Linux as I use Ubuntu and that’s based on Debian.

Enter Jeff Doozan’s website, http://www.doozan.com . They have posts and pages about specifically what I wanted to do, Debian on GoFlex Net and other ARM devices such as Pogoplug and Zyxel NSA320. It all sounded oh so simple in my research.

But it’s never as straightforward in real life is it ?

It’s taken me a lot of hours, weeks in fact to get to the point of having stable GoFlex Net with Debian Linux, OpenMediaVault and RAID1. And many many failed attempts !

This is a guide for how to do it, and sharing what I learn’t along the way. If you want to build a similar device, then you’ll be able to using this guide, the EASY WAY, avoiding the mistakes I made, saving time and stress.

Obviously if you follow this guide you do so at your own risk, and it’s your responsibility to make sure that you’re doing things correctly, not mine. GoFlex Net can be broken (aka bricked, foobared, door-stopper) if you don’t use care and caution, ESPECIALLY when doing anything to do with the inbuilt NAND flash memory. Take care in what you’re doing and you won’t have a problem. Check everything carefully and check the references I’ve given to external sites.

What you’ll need to do this

Don Charisma, doncharisma.com Toolbox
Tools

Mandatory

Seagate GoFlex Net

USB Flash Drive (not camera card eg SD), I use Sandisk Cruzer and Cruzer Fit

Ubuntu or other flavour of Linux, either running direct on a PC/Laptop or in a Virtual Machine

An SSH client program, Ubuntu has one or on Windows I use BitVise SSH Client.

Some time and patience and willingness to learn. A little bit of Linux knowledge extremely beneficial.

Optional

gparted GUI partion editor on Ubuntu

Portable/2.5″ Hard Drive(s) – Seagate Hard Drives recommended, Backup Plus 1TB are a good fit and cheap at around £50 each

STEP 1 – Get ssh access to GoFlex NetThis is how we do it

  • STEP 2 – Make a bootable “rootfs” on USB drive
  • STEP 3 – Update the GoFlex Net inbuilt bootloader to boot from USB
  • STEP 4 – First boot from USB
  • STEP 5 – Install rescue system (optional but HIGHLY recommended)
  • STEP 6 – Install netconsole (optional but HIGHLY recommended)
  • STEP 7 – Install OpenMediaVault to USB
  • STEP 8 – Migrate your rootfs to SATA hard drive (optional)
  • STEP 9 – Setup RAID1 (optional)
  • STEP 10 – Do some cool stuff with the LEDS (optional)
  • APPENDIX – for reference, chrooting for kernel upgrade/change or troubleshooting

STEP 1 – Get ssh access to GoFlex Net

In order to do anything with your GoFlex Net you’ll need to get command line access via ssh to the “root” user.

See GR3YH0UND’S BLOG on Prepare SSH Connection

Basically, you’ll probably need to go to  pogoplug website , register yourself and your device and switch ssh access on in the “Security Settings” page. You could try connecting via ssh on your GoFlex Net fresh out of the box, would save you a step.

The login for your GoFlex Net is either “root/stxadmin” or “root/ceadmin” (pogoplug website)

In order to connect to GoFlex Net command line on ssh you’ll need to either install an ssh client such as BitVise SSH (on Windows) or use ssh from a terminal window on Ubuntu (or however you want to do it !)

You’ll need to IP address for your GoFlex Net, it will get one automatically from your router. You might see this if you explore the network in Windows (will have a name like “FADS265D34”, or in your router’s list of DHCP clients with an IP address on. For mine I have my router setup so it issues the same IP address to my GoFlex Net.

Once you’ve got ssh access, you can move to next step.

STEP 2 – Make a bootable “rootfs” on USB drive

First thing to note is that GoFlex Net is NOT supported with standard Debian kernel, in my experience it just won’t boot. Also using either the Debian Squeeze or Wheezy install scripts from Doozan.com don’t work and/or are broken, so just don’t waste the hours trying, I did, and all I got was tired, frustrated and annoyed.

The MOST RELIABLE way I found of creating a bootable USB drive is to use rootfs from davygravy’s post on doozan.com download his package here. You only need the “kirkwood-debian_rootfs.img” file, ignore the other files.

# to extract the archive in Ubuntu (or use WinRAR in Windows)
tar -xvzf NSA320quickstart.tar.gz

Next we need to prepare the USB Flash drive. Personally I reckon ext2 partition format is the best to use for flash drive. I think because of it’s age it’s the simplest and least likely to cause problems.

To do this I used gparted GUI on Ubuntu, you’ll need to :

1. locate your USB drive in dropdown menu, will probably be /dev/sdb or /dev/sdc, be sure you’ve selected the correct drive !

2. delete existing paritions

3. create new partitions, I’d suggest first partition ext2 around 1GB (min 256MB) label “rootfs”, second parition linux swap 256MB label “swap”. You can move and resize these later if you so wish.

Now we need to put the rootfs onto the flash drive.

# On your Ubuntu/Linux machine , suppose your drive's partion1 /dev/sdb1 is mounted at /mntmount /dev/sdb1 /mnt
sudo -s
mkdir /media/debian
mount -o loop <path-to-kirkwood-debian-image_debian.img> /media/debian
cp -a /media/debian/* /mnt
umount /mnt
umount /media/debian
# where /mnt is the mount point of the USB drive

That’s the USB drive created, with *working* rootfs, debian 3.3.2 kernel, Debian Squeeze distibution.

SIDE NOTE: It’s possible to upgrade squeeze to wheezy on this installation with the 3.3.2 kernel. And I’ve also managed to install the 3.10.4 kernel by bodhi here. PLEASE DON’T UPGRADE KERNEL OR SQUEEZE TO WHEEZY IF YOU WANT TO INSTALL OPENMEDIAVAULT, Debian squeeze is needed by OpenMediaVault and the 3.3.2 kernel works fine.

STEP 3 – Update the GoFlex Net inbuilt bootloader to boot from USB

We need to do this so that we can boot from a USB flash drive, instead of being locked into Pogoplug software.

Update uboot

Be sure that you’re ready and prepared for this stage as will actuall *WRITE* to the flash memory in the GoFlex Net. If you don’t do it right you could brick your device here. So make sure power is on and will stay on during the procedure

# Procedure from "http://projects.doozan.com/uboot/
cd /tmp
wget http://projects.doozan.com/uboot/install_uboot_mtd0.sh
chmod +x install_uboot_mtd0.sh
./install_uboot_mtd0.sh

Restore old uBoot

This step is only for reference, DON’T ACTUALLY RUN IT NOW. Should you wish to revert to the original GoFlex Net/Pogoplug software, then you can use this procedure.

# Restore original uBoot
wget http://download.doozan.com/uboot/files/uboot/uboot.mtd0.goflexnet.original.kwb
flash_erase /dev/mtd0 0 4
nandwrite /dev/mtd0 uboot.mtd0.goflexnet.original.kwb

STEP 4 – First boot from USB

Take the USB Stick you’ve made and put into the USB port of the GoFlex Net. You’ll need to power cycle the device.

If all’s well you should see the GoFlex get an IP address from your router. In my router I have the IP address of the GoFlex Net mapped to a static IP address via the MAC address.

With the IP address you can see from router (or you’ve mapped against MAC address), connect via SSH client, login with “root/root”.

Once your in you should see a prompt :

root@debian-kirkwood-wide:/#

Do a:

uname -a

this will show you the kernel version which should be davygravy’s 3.3.2

You now have a working Debian Squeeze installtion, 3.3.2 kirkwood kernel.

If you’re NOT ABLE TO BOOT, then it’s possible BUT unlikely that you’ve bricked the device in STEP 3, if this is the case you’ll need to get more involved with fixing this, and may involved UART or serial booting. I don’t have any experience with this, so you’ll need to sort this out, help can be found at forum.doozan.com, either post a question or look at other’s questions.

More likely other reason why you’re not able to boot are that your flash drive just doesn’t work for whatever reason. I’ve had nothing but headache trying to boot from SD memory cards in generic reader, so not a good idea. From what I understand SD cards expect to be used with FAT16/FAT32 format and don’t like working with any other filesystem, which seems to explain the problems I’ve had. I mentioned earlier that best results I’ve had from Sandisk Cruzer or Cruzer fit. These aren’t the fastest devices but they do boot and seem to hold a Debian installation.

Another reason is that you’ve not created the rootfs properly. So I’d suggest try another flash drive and repeat the procedure for creating the rootfs on your USB drive.

STEP 5 – Install rescue system (optional but HIGHLY recommended)

If you don’t install the rescue system, then you’ll be forced to boot from the USB drive all the time. If the USB drive fails to boot then you will have to make another one to get into the device.  My experience is that the USB drive installations will break after you’ve spent hours and hours configuring and installing. If you want any chance of repairing then booting into debian without a bootable USB drive would be a good idea. So this and a lot of frustrating hours are why I recommend you install the rescue system. For my purposes V2 rescue work fine. The procedure is described here by Jeff Doozan.

IMPORTANT, this procedure will write to the Flash in your GoFlex Net, so have everything ready advance and make sure power won’t go off.

So either you’re in your ssh from USB drive that’s booted your davygravy’s rootfs or you need to boot into it, then :

cd /tmp
wget http://jeff.doozan.com/debian/rescue/install_rescue.sh
chmod +x install_rescue.sh
./install_rescue.sh

Reboot without the USB stick attached and you should boot into the rescue system. Then reboot with the USB attached and you should boot into davygravy’s rootfs.

STEP 6 – Install netconsole (optional but HIGHLY recommended)

Without netconsole, you can’t monitor the boot process (well at least the bits up until Kernel is loaded). It’s a bit of extra effort, but it will help you debug if the USB stick is not booting. Netconsole procedure is described by Jeff Doozan here – http://forum.doozan.com/read.php?3,14,14

Netconsole

On the Goflex NET, you’ll need to setup some environment variables for the uBoot loader. These are written into the internal flash, so be VERY VERY careful you get this right. CHECK, DOUBLE CHECK and TRIPLE CHECK anytime you are flashing internal flash or doing anything with it.

On the GoFlex Net:

# serverip is the machine that you will monitor on, ipaddres is the address of your GoFlex Net
# my Ubuntu machine I use to monitor is 10.10.10.203 and my GoFlex Net is 10.10.10.231

fw_setenv serverip 10.10.10.203
fw_setenv ipaddr 10.10.10.231
fw_setenv if_netconsole 'ping $serverip'
fw_setenv start_netconsole 'setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;'
fw_setenv preboot 'run if_netconsole start_netconsole'

On the desktop machine (Ubuntu – 10.10.10.203)

nc -u -l 6666 &
nc -u 10.10.10.231

You’ll need to reboot in order for these settings to take effect. I’ve got into the habit of doing this :

# for some linux installations
sync
shutdown -h now

# or on some other linux installations
sync
halt

Reason for doing this is to make sure any cached data is written to disk and propertly shutdown the operating system.

STEP 7 – Install OpenMediaVault to USB

OpenMediaVault installation was fairly straightforward, although had to do some digging around on forums ! This thread on OMV forum is now out of date, but forms that basis of how to do it.

Update sources.list file

It’s good practice to do this on any installation of Debian, and will speed up any installations and updates because hopefully you’ll find mirrors that are close to you. Please refer to Debian worldwide mirrors full list to get yours.

vi /etc/apt/sources.list

This is what yours will look like:

root@debian-kirkwood-wide:/# cat /etc/apt/sources.list
deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free
deb http://ftp.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.debian.org/debian/ squeeze main contrib non-free

As I’m based in Asia, this is what mine looks like after editing:

deb http://ftp.th.debian.org/debian-security/ squeeze/updates main contrib non-free
deb-src http://ftp.th.debian.org/debian-security/ squeeze/updates main contrib non-free
deb http://ftp.th.debian.org/debian/ squeeze main contrib non-free
deb-src http://ftp.th.debian.org/debian/ squeeze main contrib non-free

Now we need to update the out Debian installation to latest version of Squeeze, and install xmlstartlet. For some reason the xmlstarlet used by OMV comes from debian Wheezy (but OMV isn’t stable on Wheezy), so we also need to install xmlstarlet, then upgrade it to a later version manually.

apt-get update
apt-get upgrade
apt-get install xmlstarlet

Get xmlstarlet 1.5.0-1 and install it:

wget http://ftp.us.debian.org/debian/pool/main/x/xmlstarlet/xmlstarlet_1.5.0-1_armel.deb
dpkg -i xmlstarlet_1.5.0-1_armel.deb

Now do a HARD reboot, sync, shutdown and then power cycle.

sync
shutdown -h now

You should have rebooted and be back at your ssh prompt.

Now I change the hostname, my GoFlex Net is called “GoFlexNet1”, you can call yours what you want.

root@debian-kirkwood-wide:~# cat /etc/hostname
debian-kirkwood-wide
root@debian-kirkwood-wide:~# echo "GoFlexNet1" > /etc/hostname
root@debian-kirkwood-wide:~# cat /etc/hostname
GoFlexNet1

Now we update our hosts file and tell it what out IP address is. I’d recommend setting this in your router as mentioned earlier. If you want a static IP address then you’ll need to configure your /etc/network/interfaces or do it via the OMV interface.

root@debian-kirkwood-wide:~# cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

root@debian-kirkwood-wide:~# vi /etc/hosts
root@debian-kirkwood-wide:~# cat /etc/hosts
127.0.0.1 localhost
10.10.10.231 GoFlexNet1
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Now do a HARD reboot, sync, shutdown and then power cycle.

sync
shutdown -h now

Now we need to setup /etc/apt/sources.list.d/openmediavault.list so that OMV knows where to look for it’s files, install the OMV keys, install OMV and restart the Apache server.

OMV installer will ask questions, so don’t stray too far:

  • about mail server (I didn’t bother will setup later)
  • RAID arrays, I don’t think GoFlex can boot from RAID so no point starting the RAID early, if you are planning to use RAID then you’ll need to start the arrays, OMV will do this for you
echo "deb http://packages.openmediavault.org/public fedaykin main" > /etc/apt/sources.list.d/openmediavault.list

# the first apt-get update will throw an error, thats why we install the openmediavault-keyring

apt-get update
apt-get install openmediavault-keyring
apt-get update
apt-get install openmediavault
service apache2 restart

DO NOT REBOOT, YET ! First we need to setup a couple of things, especially re-enabling the now disabled ssh.

First go to your internet browser firefox chrome etc, my page is http://10.10.10.231

Login to the OMV web interface using “admin/openmediavault”, and setup the following :

  • enable ssh /permit root
  • general settings – session timeout 0 (nothing more annoying than having to re-login every 5 minutes)
  • date & time

*Shutdown using OMV web interface and power cycle.*

Go to browser and again login to OMV web interface and do:

  • update manager – update to 0.4.37

*Shutdown using OMV web interface and power cycle.*

At this stage I attached an SATA hard disc and tried to boot from USB. No joy, doesn’t boot. This problem is described here reply here. Basically uBoot and the kernel loaded from USB disagree on where the rootfs is. The solution provided by bodhi is to force uBoot and the Kernel to boot from a partition called “rootfs”.

So remove the SATA drive and reboot, then do this :

# USE CAUTION, updates internal flash
fw_setenv usb_bootcmd 'run usb_init; setenv usb_root LABEL=rootfs; run usb_set_bootargs; run usb_boot'

Now insert SATA drive again

*Shutdown using OMV web interface and power cycle.*

You should now have a working OpenMediaVault on your USB Flash Drive, that you can boot with SATA drives attached.

You may wish to leave configuration like this and continue booting off the USB flash drive, that’s fine if you trust the USB flash drive and do regular backups of the USB drive. Advantage of booting from USB flash drive is that you can use the whole of your SATA drives for data and setting up RAID is slightly simpler, because you can RAID the whole drives. Disadvantage is that I’m not convinced of the longevity of a USB flash drive for linux installations, so it’s possible it will break at some point. Also your USB port is now tied up, so attaching anything else to USB port is more complicated.

One solution to make this more reliable would be to boot off a USB hard disc, and you’d still be RAID the whole drives, but USB port still occupied.

My solution is to boot off of one of the SATA attached drives, which is what I’m not going on to describe next. Downside is that RAID has to be manually setup and so do the partitions.

STEP 8 – Migrate your rootfs to SATA hard drive (optional)

I’ve already given my reasons for wanting to do this, basically I believe that this is more stable and reliable.

First we need to partition the drive(s), I used gParted GUI in Ubuntu, but Linux heads feel free to command line it. I’ve made my rootfs just a bit smaller than a 4GB flash drive so that I could just copy it onto flash drive for backup, and also allows for logs to grow and any updates that may happen later. So here’s my layout

Partition1 -> size=3800MiB -> format to ext3 -> label = “rootfs_hd”

Partition2 -> size=512MiB -> format to linux-swap -> label=”swap”

Partition3 -> size=The rest of the disk -> unformatted -> label=””

Do the same for both disks if you have two, partition1 on second disk I labeled “backup_rootfs_hd”. Also make a mental note of which disk is “master” (the one to boot from) and which is “backup”

Now you need to copy the rootfs that you’ve created on your USB flash drive to your “master” hard disk. Ubuntu is quite helpful in mounting drives so they may already be mounted, otherwise you’ll need to do it manually

# assuming "master" hard disk rootfs is /dev/sdc1 and usb flash drive rootfs is /dev/sdd1
mkdir /media/hd_rootfs
mkdir /media/usb_rootfs
mount /dev/sdc1 /media/hd_rootfs
mount /dev/sdd1 /media/usb_rootfs
cp -a /media/usb_rootfs/* /media/hd_rootfs

So you should now in theory have a bootable hard drive. BUT there is one step left, your uBoot isn’t yet configured for SATA booting. How to set this up was described by varkey here. Basically we need to set some variables for uBoot, so again be careful doing this.

fw_setenv sata_boot 'ide reset; mw 0x800000 0 1; ext2load ide 0:1 0x800000 /boot/uImage; if ext2load ide 0:1 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; else bootm 0x800000; fi'
fw_setenv sata_bootcmd 'run usb_set_bootargs; run sata_boot'
fw_setenv bootcmd 'usb start; run force_rescue_bootcmd; run ubifs_bootcmd; run usb_bootcmd; usb stop; run sata_bootcmd; run rescue_bootcmd; run pogo_bootcmd; reset'

Now that you’ve done that you can try to boot from the hard disk and it should work.

STEP 9 – Setup RAID1 (optional)

If you’ve got this far then well done !

My discs where mounted /dev/sda /dev/sdb. The partitions I wished to RAID1 where around 950GB each /dev/sda3 /dev/sdb3. I used <a href=”http://linuxconfig.org/linux-software-raid-1-setup”>this guide</a> here loosely as a reference for this, but note his is quite a different setup.

First of all find out what your partitions are labeled, and have a look where rootfs_hd is mounted which well tell you which is master and which is backup.


# list all partitions on disks
sfdisk -l

# check mounts, where rootfs is mounted
mount -l

Now the unformatted rest-of-disk partitions we created earlier need to have their partition type switched to “linux raid autodetect”, and then list all partitions again so we know they’ve been updated.

#change partition type of partition 3 to fd (linux raid autodetect)
sfdisk -c /dev/sdb 3 fd

#change partition type of partition 3 to fd (linux raid autodetect)
sfdisk -c /dev/sda 3 fd

# list all partitions on disks
sfdisk -l

Now you’re going to create the array. Once created it will immediately do a resync, and this will take 3-4 hours. I didn’t think about it ahead of time so I had no choice but to wait, and I reckon it’s not a bad idea to let the RAID system check the disks, as I don’t want a bad disk in my array. If you know what you’re doing and are confident about it, consider using the –assume-clean flag, which avoids doing a resync and saves time. The downside I reckon is that you don’t get your disks checked. Up to you, see here for more details.

# create raid device (consider using --assume-clean)
#
mdadm --create md1 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3

mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md/md1 started.

RESYNC will take around 3-4 hours for 1TB drives, so be patient, montior and don’t reboot or poweroff whilst it’s being done. You can continue doing things, but anything you do on the drives will temporaily pause the resync and it will take longer. You can continue setting up filesystem etc, just don’t copy a bunch of data in until the resync has finished.

As you can see my array is called /dev/md/md1 and also it seems /dev/md127 (not quite what I was expecting !). It seems that these device iditifiers are interchangable, OMV will use the md127 one.

These commands will tell you about the array and you’ll be able to find your md127 device in “ls /dev”

ls /dev

#basic details of raid
mdadm /dev/md127

#detailed details of raid
# http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/
mdadm --detail /dev/md127
#or
cat /proc/mdstat

Next we need to make the filesystem on the array

#make the file system ext3

mkfs.ext3 /dev/md127

Lastly as described here , label your array, I’ve called mine “raid1”

#label the raid 1 array
#
tune2fs -L raid1_data /dev/md127

Your RAID1 array should now be configured and “seeable” in OMV web interface. Obviously once it’s finished resyncing power down using OMV and power cycle. Then you can start using it:)

Optional Bonus !

If you want to manually backup your live rootfs from one master to backup, you already have partitions in place for this. Use this procudure :

#backup rootfs
mkdir /tmp/backup
mount /dev/sdb1 /tmp/backup
cd /tmp/backup
mkdir rootfs_backup
rsync -aAXv /* /tmp/backup/rootfs_backup --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

This means that in the event that your master disk dies then you have a backup that you can boot into on the “backup” disc. I don’t know how best to automate this, and whether wise to automate. Probably best to run prior to any updates, and maybe once a week or so.

STEP 10 – Do some cool stuff with the LEDS (optional)

Getting the LEDs giving some feedback about what’s going on, seemed to be a nice-to-have, but since implimenting this I’ve found that it’s invaluable. So I’d suggest strongly consider doing this. There’s several things I wanted to get feedback on :

1. Disk and CPU activity

2. When is it “safe” to disconnect the power at shutdown

3. Network activity

I wasn’t able to get network activity working, I think it may be possible with the “sar” command that comes with iostat. The method with iptables didn’t work, I’m assuming because the davygravy kernel I’m using isn’t setup for it. Happy for any readers to report back how to do this.

For Disk and CPU, the iostat command is perfect. I did a post on Doozan forum here –“by Don Charisma – LEDs on GoFlex Net”. I use an awk command to process the output of iostat, and then feed this to the LEDs. You could change the position of the LEDs to suit your own particular scenario.

Here’s what the LEDs are assigned to :

doncharisma, don charisma, Seagate GoFlex Net LEDs
Seagate GoFlex Net LEDs

Here’s how we do it:

You’ll need to install the systat package :

# for iostat
apt-get install sysstat

Now we need to edit the /etc/rc.local file like so :

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# For LED control on Seagate GoFlex Net that's been Debianised
# (c) Don Charisma 2013

iostat -z 1 | awk -W interactive '
BEGIN { sda="none";
		sda_prev="none";
		sdb="none";
		sdb_prev="none";
		md1="none";
		md1_prev="none";
		cpu20="none";
		cpu20_prev="none";
		cpu40="none";
		cpu40_prev="none";
		cpu60="none";
		cpu60_prev="none";
		cpu80="none";
		cpu80_prev="none";
		cpu_utilise=0
	  }
# if hard drives or RAID active, set flag for LED
/sda/ {	sda="timer"; }
/sdb/ {	sdb="timer"; }
/md1/ { md1="default-on"; }
# Find the line with CPU info, pick the last number = percentage idle
# subtract it from 100 to give CPU utilisation
/\..*\..*\..*\..*\..*\..*/  { cpu_utilise = 100 - $6;
				if (cpu_utilise >= 20.0) { cpu20="default-on"; }
				if (cpu_utilise >= 40.0) { cpu40="default-on"; }
				if (cpu_utilise >= 60.0) { cpu60="default-on"; }
				if (cpu_utilise >= 80.0) { cpu80="default-on"; }
				}
/Device:/ { if (sda != sda_prev)
				{ print sda > "/sys/class/leds/status:white:left0/trigger";
			      fflush( "/sys/class/leds/status:white:left0/trigger" );
				}

			if (sdb != sdb_prev)
				{ print sdb > "/sys/class/leds/status:white:left1/trigger";
				  fflush( "/sys/class/leds/status:white:left1/trigger" );
				}

			if (md1 != md1_prev)
				{ print md1 > "/sys/class/leds/status:white:left2/trigger";
				  fflush( "/sys/class/leds/status:white:left2/trigger" );
				  print md1 > "/sys/class/leds/status:white:left3/trigger";
				  fflush( "/sys/class/leds/status:white:left3/trigger" );

				}
			if (cpu20 != cpu20_prev)
				{ print cpu20 > "/sys/class/leds/status:white:right0/trigger";
				  fflush( "/sys/class/leds/status:white:right0/trigger" );
				}

			if (cpu40 != cpu40_prev)
				{ print cpu40 > "/sys/class/leds/status:white:right1/trigger";
				  fflush( "/sys/class/leds/status:white:right1/trigger" );
				}

			if (cpu60 != cpu60_prev)
				{ print cpu60 > "/sys/class/leds/status:white:right2/trigger";
				  fflush( "/sys/class/leds/status:white:right2/trigger" );
				}

			if (cpu80 != cpu80_prev)
				{ print cpu80 > "/sys/class/leds/status:white:right3/trigger";
				  fflush( "/sys/class/leds/status:white:right3/trigger" );
				}

			sda_prev=sda;
			sda="none";
			sdb_prev=sdb;
			sdb="none";
			md1_prev=md1;
			md1="none";
			cpu20_prev=cpu20;
			cpu20="none";
			cpu40_prev=cpu40;
			cpu40="none";
			cpu60_prev=cpu60;
			cpu60="none";
			cpu80_prev=cpu80;
			cpu80="none";

}
END { print "The end"; }
' &

exit 0

Now for shutdown/halt LEDs, to make green LED go off and the white LEDs to blink, we need to make a change to the /etc/init.d/halt file, here’s how the section of mine that I changed looks :


# Make it possible to not shut down network interfaces,
# needed to use wake-on-lan
     netdown="-i"
     if [ "$NETDOWN" = "no" ]; then
     netdown=""
     fi

# for goflex green led
     echo none > /sys/class/leds/status\:green\:health/trigger

# for goflex white leds
     echo timer | tee /sys/class/leds/status\:white\:*/trigger > /dev/null

log_action_msg "Will now halt"
     halt -d -f $netdown $poweroff $hddown

}

That’s it your done, enjoy:)

APPENDIX – for reference, chrooting for kernel upgrade/change or troubleshooting

This procedure was posted by tuxuser in doozan.com forums here. I’ve made some slight edits.

In the event that your USB flash drive won’t boot, you should still be able to get your GoFlex Net running via the rescue system. You can then chroot into the rootfs on your USB drive, and perhaps make repairs, look at logs, maybe fix the problem.

Also if you want to change kernel, it has to be done via chroot, you can’t change a live kernel. The procedure here is for changing kernel, if you just want to chroot into your installation, stop once you’re chroot’ed.

Once you’ve booted into the rescue system, this would be “a” procedure to change the kernel.

# Mount roofs
mkdir /tmp/wheezy
mount /dev/sda1 /tmp/wheezy
cd /tmp/wheezy
# Prepare chroot-environment
mount -t proc proc proc/
mount -t sysfs sys sys/
mount -o bind /dev dev/
mount -t devpts pts dev/pts/
chroot /tmp/wheezy /bin/bash

# Now here we go with the kernel install
#Dependencies for a https wget
apt-get update apt-get install dialog ca-certificates openssl
wget https://dl.dropbox.com/sh/nwt2kbvetvgvroo/661x9b5PLg/Kirkwood/kernel/linux-image-3.3.2-kirkwood-dg_1.1_armel.deb?dl=1 -O linux-image-3.3.2-kirkwood-dg_1.1_armel.deb

# Safety
cp -r /boot /boot-backup

# flash-kernel never installed for me successfully for me -> removing postinstall-script, otherways update-initramfs will bug around
rm /etc/kernel/postinst.d/zz-flash-kernel

# Install Kernel
dpkg -i linux-image-3.3.2-kirkwood-dg_1.1_armel.deb

# Build proper uboot images (neccessary because uBoot has to boot from images)
cd /boot
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-3.3.2 -d vmlinuz-3.3.2-kirkwood-dg uImage
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-3.3.2-kirkwood-dg -d initrd.img-3.3.2-kirkwood-dg uInitrd

# Exit chroot
exit
# Unmount chroot-environment
umount /tmp/wheezy/dev/pts
umount /tmp/wheezy/dev
umount /tmp/wheezy/sys
umount /tmp/wheezy/proc cd /
umount /tmp/wheezy

#Reboot
init 6

TO DO

I’d like to get LEDs to respond to network traffic. I saw  a post here relating to this, well I couldn’t get the iptables thing to work, not sure why as yet. Wasn’t able to think of another Linux tool to use to detect network traffic or not, since the iptables command gives an error. Perhaps the sar tool would do.

I think I will also try changing the RAID system to RAID10,f2 as described here. I wasn’t aware when I started this of RAID10,f2, which is why I used RAID1. RAID1 has the possibility of being faster than a single disk for reads but in practice for a single client connection, you’re reading off a single disk, probably the master. RAID10,f2 however can be setup on just 2 discs, and the data is organised in such a way that data can be read off of both disks at the same time to serve a single client.

Also like to get the swap partitions RAID’ed, maybe the rootfs, although there are reasons for not RAIDing the rootfs.

I might build my second unit the same, but use NTFS drives mirrored using rsync, for flexibility in plugging them into my Windows machines.

The netconsole is great up until the kernel starts booting, but then I’m not getting any messages from the booting system. It’s possible to get these boot messages sent via netconsole, but requires fiddling with the kernel as described here. I think I will try and add this as difficult to debug boot problems without it.

Confusion and Conclusion

In total I must have spent a month working on this, it looks easy but isn’t. Hopefully this guide has taken out a lot of the donkey work and hair pulling, and you may be able to do it in a day:)

Building a NAS that’s on a par with devices costing far more is entirely possible. Just a case of persistence and willingness to learn.

My GoFlex NET has now been running a couple of weeks and seems stable enough with OpenMediaVault and my two 1TB RAID1 disks. Performance wise I’m happy enough, only bottleneck I’ve found so far is the Samba daemon which is using a lot of CPU when I copy files over the network onto the RAID array. But still I’m getting around 20MB/s which is OK.

Thank you to all the guys at doozan.com forum, Jeff, vlad, davygravy, bohi, varkey, Gr3yh0und and others, your forum and blogs have been invaluable resources. Also a big thanks to the developers of OpenMediaVault, I really hope that their project is going to continue and get better and better, amazing work.

I’m happy to answer any questions, if I can, but best place to ask really is register at doozan.com forum and ask there, other people may have same problem and be looking on Jeff’s forum.

If anyone from Seagate reads this, then yes I’d be happy to work as a consultant, build for you some decent software for your products.

Let me know of any feedback or errors, so that I can update this post.

Happy Debian’ing and NAS’ing:)

Resources and Sources

DonCharisma, LEDs on GoFlex Net – http://forum.doozan.com/read.php?2,13561

Davygravy’s rootfs, (Zyxel NSA320 : basic support for Debian installation) – http://forum.doozan.com/read.php?2,7806

How to extract a .tar.gz file – http://askubuntu.com/questions/25347/what-command-do-i-need-to-unzip-extract-a-tar-gz-file

Gr3yh0und on Debian/GoFlex Net – http://morschi.com/2012/10/05/seagate-goflex-net-debian/

Rescue System v2 – http://forum.doozan.com/read.php?4,3896

Dockstar Rescue Linux – http://www.rudiswiki.de/wiki9/DockStarRescue

OMV on Pogoplug – http://forums.openmediavault.org/viewtopic.php?f=12&t=514

Debian Sources Mirrors Worlwide – http://www.debian.org/mirror/list-full

(In German) OMV on GoFlex – http://forums.openmediavault.org/viewtopic.php?f=12&t=2552

USB when hard drives attached boot issue, http://forum.doozan.com/read.php?3,9398

Varkey on booting Debian from SATA GoFlex Net – http://www.varkey.in/2011/06/boot-debian-from-sata-seagate-goflex-net/

A post about setting up linux RAID1 – http://linuxconfig.org/linux-software-raid-1-setup

A post about Inital Array Creation – https://raid.wiki.kernel.org/index.php/Initial_Array_Creation

Preventing against a failing disk, RAID 10,f2 – https://raid.wiki.kernel.org/index.php/Preventing_against_a_failing_disk

How to label linux partitions – http://www.cyberciti.biz/faq/linux-partition-howto-set-labels/

Chrooting – http://forum.doozan.com/read.php?2,9256,9957#msg-9957

Howto: Getting network LED working with firewall – http://forum.doozan.com/read.php?2,4638,4645

Howto: Create Debian rootfs image for your GoFlex/Dockstar via debootstrap on a Ubuntu/Debian PC – http://forum.doozan.com/read.php?2,5986,6221

GoFlex Net: Boot from USB while SATA disks connected – http://forum.doozan.com/read.php?3,9398

Full System Backup with rsync – https://wiki.archlinux.org/index.php/Full_System_Backup_with_rsync

DockstarDebianSqueeze (In German) – http://www.mikrocontroller.net/articles/DockstarDebianSqueeze

Front LED (In German) – http://www.mikrocontroller.net/articles/Dockstar#Front_LED_ansprechen

Newer uBoot as workaround to 3.2 kernel problem – http://forum.doozan.com/read.php?3,6965

OBIHOERNCHEN’S BLOG, Posts tagged GoFlex (see for LED info) – http://obihoernchen.net/wordpress/tag/goflex/

Redirect stdin stdout to multiple files – http://www.unix.com/unix-dummies-questions-answers/124609-redirect-stdin-stdout-multiple-files.html

How can I send stdout to multiple commands? – http://unix.stackexchange.com/questions/28503/how-can-i-send-stdout-to-multiple-commands

want leds to change to hard drive activity – http://archlinuxarm.org/forum/viewtopic.php?t=2899

Ten(ish) steps to set up a dockstar – http://archlinuxarm.org/forum/viewtopic.php?f=18&t=289

Pogoplug root password (Linux Developers) – http://download.pogoplug.com/dev/linux.html

Review Seagate FreeAgent GoFlex Net Media Sharing Device – http://www.notebookcheck.net/Review-Seagate-FreeAgent-GoFlex-Net-Media-Sharing-Device.37033.0.html

Linux Kernel 3.10.4 Kirkwood package and rootfs (Non Flattened Device Tree) – http://forum.doozan.com/read.php?2,12096

Use netconsole to troubleshoot uBoot without a serial cable – http://forum.doozan.com/read.php?3,14,14

Kernel install fails with Wheezy script – http://forum.doozan.com/read.php?2,9256

Howto: Getting network LED working with firewall – http://forum.doozan.com/read.php?2,4638

[How To] Use netconsole to troubleshoot Debian kernel booting – http://forum.doozan.com/read.php?2,9522

Proper LED Setup – http://dev.shyd.de/2012/01/kernel-3-1-10-ready-to-use/

27 thoughts on “Build your own Pro NAS – Seagate Goflex Net with Debian Linux, RAID1 and OpenMediaVault

  1. After reading this guide, I bought a Seagate Go Flex device in May 2016 for just 12€ (with delivery). And there the problem starts, after I registered the device I tried to enable SSH access by official method (i-e from Pogoplug website) but sadly now under the security settings on webpage there is no option to enable SSH.

    So I searched on internet and found a back door method, which I would like to share with you people. (Credit to some awesome people out there).

    From any Linux computer (on same network as your Pogoplug) run the below curl command.

    curl -k “https://root:ceadmin@[PogoplugIPAddress]/sqdiag/HBPlug?action=command&command=dropbear%20start”

    Replace [IPAddress] (including brackets) with your PogoPlug’s IPAddress.

    e-g curl -k “https://root:ceadmin@10.10.10.10/sqdiag/HBPlug?action=command&command=dropbear%20start”

    That may spit out some strange looking text, that’s it you got it.

    After that you can login to your Pogo plug device from any SSH client using
    User Name: root
    Password: stxadmin (if that don’t work try ceadmin)

    Note that the backdoor method is not persistent (SSH access not available after a reboot).

    Cheers
    Ishwar

  2. This was posted on http://DonCharisma.com by BKrueger

    Hi,
    massive thanks for your work!

    I Just want to give a short feedback – since i had some trouble using the LEDs =).

    1. as some guys said already, there is one command missing:
    fw_setenv arcNumber 3089
    I set this var in the very end (before trying to install a sata drive) but i guess, you are not able to use sata/LEDs (you can control only the green one) before setting this variable correctly!

    2. The script which is used to show the cpu usage is not working properly. I just fixed the little awk bug, so that now only the real cpu idle value is taken into account.

    Just replace this part
    >>
    # Find the line with CPU info, pick the last number = percentage idle
    # subtract it from 100 to give CPU utilisation
    /..*..*..*..*..*..*/ { cpu_utilise = 100 – $6;
    if (cpu_utilise >= 20.0) { cpu20=”default-on”; }
    if (cpu_utilise >= 40.0) { cpu40=”default-on”; }
    if (cpu_utilise >= 60.0) { cpu60=”default-on”; }
    if (cpu_utilise >= 80.0) { cpu80=”default-on”; }
    }

    # Find the line with CPU info, pick the last number = percentage idle
    # subtract it from 100 to give CPU utilisation
    /avg-cpu:/ {
    getline;
    cpu_utilise = 100 – $6;
    if (cpu_utilise >= 20.0) { cpu20=”default-on”; }
    if (cpu_utilise >= 40.0) { cpu40=”default-on”; }
    if (cpu_utilise >= 60.0) { cpu60=”default-on”; }
    if (cpu_utilise >= 80.0) { cpu80=”default-on”; }
    }
    <<

    cheerio and massive thx !!!

  3. Tried it, up to the point that I install openmediavault (apt-get install openmediavault) but I got an error message saying that :

    The following packages have unmet dependencies:
    openmediavault : Depends: php5-pam but it is not going to be installed
    Depends: scponly but it is not installable

    1. Hey xavier, great let me know if I missed anything:) … I’m mainly Windows for my desktop and laptops, tend to use linux for stuff that’s easier in linux than in windows, if that makes sense… I’m liking how linux is progressing with Ubuntu, a commercial sponsor and a free operating system…I wish they hadn’t taken the menu out, just means I have to fiddle around removing default UI and going back to the orginal one…although a lot of the time I don’t use the menus I like to know they are there, if that makes sense !

      Cheers

      Don Charisma

      1. Actually, if you are referring to the OLD menu (used in version 10.04 and earlier), you can get them in ubuntu 13.04 as well. Google “gnome fallback mode 13.04”.

      2. Not sure when it changed, my last install was a 12.10 I think and that had a new UI. Problem I have is, if there’s a menu system I can fail back to I’m fine, but having to remember what everything is called and search for hurts my head, doh ! … I found a hack somewhere to banish the desktop and put the old one back (well it’s selectable at login), so no problem …

        I like to flirt with different OS’es, recently had OSX running in VMWare, courtesy of the souldev team, like to try that on my laptop, bareback, bare metal or whatever:)

        My desktop has 4 screens and I was worried Linux wouldn’t drive them, but guess what XBMC works fine, so I’m assuming Ubuntu would too, but haven’t actually tried yet, a project for another day.

        Also have had some fun with BSD via pfSense router software and managed to get the multi-wan going. I love pfSense, but it likes to run on powerful hardware, so have settled OpenWRT on my router…

        Good to chat,

        Don Charisma

      3. Ubuntu runs multiple Screens easily. It recognizes them on boot. I myself use 2 screens (thinking of a third as the angle of the screens seems too absurd).

        Mac OSX was a great OS, but it is really difficult to install (on a normal PC/laptop), maintain, hack or modify. In short, if you want to run OSX, you gotta be prepared for a number of different codes, hacks and stupid things just to get it up and running, and after that, live a closed life (as hardly anyone uses OSX). Ubuntu and OSX are one of the least used OS’s around the globe, but even Ubuntu beats OSX by a percent (a percent of a hundred billion is not less).

        Here is the code for enabling fall back.
        sudo gnome-session-fallback

        Then you can simply select the Fall Back mode from the login screen (pretty easy, as you can just click the small round icon on the top-right of your account at the login screen).

        I have never looked back at Windows, ever since I parted with it (actually “partitioned” with it) and went the Ubuntu way of Operating Systems. It is faster, customizable, virus free and does not require me to download puny drivers for my PC before I plug a device.

        Although I still have that Windows on a partition in my HDD, so I select it from the boot whenever I want to use apps that are made exclusively for Windows.

      4. Yes 3 widescreens these days is quite a lot of desk real estate, my 2×2 covers the entire back of my desk. When I’m thinking of improvements, I reckon four (or two) independent computers that could “Join” their desktops would an interesting solution:)

        I’ve had OSX up and running in VMWare, and quite liked it, although not sure I’d want it as my mainstay desktop/laptop operating system. At the time I was having problems with iTunes match and this did provide a solution.

        THanks for the command for desktop fallback, next install I will use that.

        Over the years I’ve become more and more cautious of making changes to my computers, because of the almost inevitable something breaks, if I must then I try to schedule when I’m not busy in the middle of something. Yesterday, I broke my own rule for installing updates and ended up with a keyboard not working due to a microsoft bluetooth driver, installed the manufacturer’s driver and problem fixed.

        Linux as mentioned already is coming of age now I think, and each time I have a problem with Windows, it’s just a little push further in that direction. With vbox and vmware ability to run guest OS’es it’s much harder to find a reason to stay with Windows. Hard for me to part from Microsoft as I’ve been using their stuff since the start. If Windows 8/9 doesn’t put the start menu back then I probably go full on Ubuntu when the time is right.

        Anyway happy Linux’ing, best wishes, Don Charisma

    1. My pleasure, and thanks for following me:)

      To get the nice code block, wrap you code in the editor with (Open Square Bracket)code language=”bash”(Close Square Bracket)

      If you want to highlight certain lines – (Open Square Bracket)code language=”bash” highlight=”6,7,8,9″(Close Square Bracket)

      Obviously replace the sqaure brackets with real thing … wordpress.com page – http://en.support.wordpress.com/code/posting-source-code/
      Wordpress.org plugin – http://wordpress.org/plugins/syntaxhighlighter/
      and
      http://www.viper007bond.com/wordpress-plugins/syntaxhighlighter/

      Keep in touch, DOn Charisma

Leave a Reply for Don Charisma

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s