Updated: 2016-02-04
Performance Note: When using JTR in a distributed MPI cluster with a central NFS server providing all configuration and code for JTR and are using a large dictionary file you may wish to avoid a huge network "surge" when all your compute nodes are pulling the dictionary file. Use something like 'salt' or 'cfengine' or even 'chef' to keep that distributed out to the compute nodes instead.
I ran into a performance problem (only on initial startup of the JTR run) it was hammering the network badly since I have a very huge dictionary file. This was only a problem though, because I chose esthetics over performance by using wifi (Netgear with DD-WRT) instead of a hardwired switch.
As you can imagine having 65 wifi transceivers (the master node (cfengine master, nfs server, MPI control node) is hardwired 100M) sitting within inches of each other doesn't work that well and the congestion is already immense with that many hosts on a consumer-grade access point.
I would need 66 switch ports however and I didn't feel like having all that wiring spaghetti for 66 computers. Perhaps some day I will extend the monolith vertically to accommodate all that but not right now.
CFEngine example for how to distribute updated dictionaries.
copy:
node_server::
/master/cfengine/root/mangled.list dest=/var/list/mangled.list
owner=root group=root mode=0644
server=$(cfsrvhost) type=checksum
Updated: 2016-02-06
I have been working with some Cubieboard 2 boards recently.
I installed Linaro to the nand (fel-mode using LIveSuit) but needed to resize nand due to the partitioning not making use of 2gb of the 4GB NAND (wasted space!)
root@cubieboard:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/nandd 2066000 960760 1013792 49% /
none 4 0 4 0% /sys/fs/cgroup
none 92784 164 92620 1% /run
none 5120 0 5120 0% /run/lock
none 463900 0 463900 0% /run/shm
none 102400 0 102400 0% /run/user
overflow 1024 0 1024 0% /tmp
root@cubieboard:~#
This is what I did:
I booted an image of Linaro 13.06 from the sd card. Then I examined the existing configuration of /dev/nand. I bought (3) boards from EBAY that had been mucked with, Android was missing and the partitioning had been mangled.
1, cat /proc/partitions
root@cubieboard2:~# cat /proc/partitions
major minor #blocks name
93 0 3989504 nand
93 1 32768 nanda
93 2 16384 nandb
93 3 32768 nandc
93 4 2097152 nandd **rootfs**
93 5 1794048 nande **wasted space**
8 0 117964900 sda
8 1 117963876 sda1
179 0 3872256 mmcblk0
179 1 12288 mmcblk0p1
179 2 3858944 mmcblk0p2
root@cubieboard2:~#
root@cubieboard2:~# nand-part|tail
BAD!
check partition table copy 3: mbr: version 0x00000200, magic softw411
BAD!
mbr: version 0x00000200, magic softw411
5 partitions
partition 1: class = DISK, name = bootloader, partition start = 32768, partition size = 65536 user_type=0
partition 2: class = DISK, name = env, partition start = 98304, partition size = 32768 user_type=0
partition 3: class = DISK, name = boot, partition start = 131072, partition size = 65536 user_type=0
partition 4: class = DISK, name = rootfs, partition start = 196608, partition size = 4194304 user_type=0
partition 5: class = DISK, name = UDISK, partition start = 4390912, partition size = 3342336 user_type=0
root@cubieboard2:~#
Once you know the exact layout of the nand partitions it is straightforward. You locate the unused space on the nand, in this
case it was partition 5 or /dev/nande.
Since we want to combine nandd and nande we simply add 4194304 (size of nandd) to 3342336 (size of nande) or 7536640
total to derive our value for the next step.
nand-part -f a20 /dev/nand 32768 "bootloader 65536" "env 32768" "boot 65536" "rootfs 7536640"
Once this completes without error, you need to run a filesystem check and then resize the newly combined partition.
root@cubieboard2:~# fsck -f /dev/nandd
fsck from util-linux 2.20.1
e2fsck 1.42.9 (4-Feb-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/nandd: 28129/123904 files (0.1% non-contiguous), 247980/524288 blocks
root@cubieboard2:~# resize2fs /dev/nandd
resize2fs 1.42.9 (4-Feb-2014)
Resizing the filesystem on /dev/nandd to 972800 (4k) blocks.
The filesystem on /dev/nandd is now 972800 blocks long.
root@cubieboard2:~#
Here you will see the results afterwards.
root@cubieboard:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/nandd 3832816 961324 2701816 27% /
none 4 0 4 0% /sys/fs/cgroup
none 92784 160 92624 1% /run
none 5120 0 5120 0% /run/lock
none 463900 0 463900 0% /run/shm
none 102400 0 102400 0% /run/user
root@cubieboard:~#
root@cubieboard:~# cat /proc/partitions
major minor #blocks name
93 0 3989504 nand
93 1 32768 nanda
93 2 16384 nandb
93 3 32768 nandc
93 4 3891200 nandd
root@cubieboard:~#
Unfortunately since this release is so old it is no longer supported on ports.ubuntu.com so
I have to make some changes to apt in order to still be able to get packages. Change /etc/apt/sources.llist from:
deb http://ports.ubuntu.com/ubuntu-ports/ raring main universe
deb-src http://ports.ubuntu.com/ubuntu-ports/ raring main universe
to
deb http://old-releases.ubuntu.com/ubuntu/ raring main universe restricted multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ raring main universe restricted multiverse
deb http://old-releases.ubuntu.com/ubuntu/ raring-security main universe restricted multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ raring-security main universe restricted multiverse
deb http://old-releases.ubuntu.com/ubuntu/ raring-updates main universe restricted multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ raring-updates main universe restricted multiverse
deb http://old-releases.ubuntu.com/ubuntu/ raring-backports main restricted universe multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ raring-backports main restricted universe multiverse
deb http://old-releases.ubuntu.com/ubuntu/ raring-proposed main restricted universe multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ raring-proposed main restricted universe multiverse
