Wednesday, April 02, 2014

OSX Development Environment Setup

I just upgraded a 2010 13” Macbook Pro with a 240GB Intel SSD 530. Started with a clean install of OSX 10.9 which I immediately updated to 10.9.2
Here are the apps I installed:
Platforms
Editors/IDE
GUI for common tools
Browsers
Utilities
  • Homebrew: package manager for OSX
  • TrimEnabler: enables SSD trim for non Apple drives
  • iTerm2: a better terminal
  • Alfred: quick search and productivity tool
  • Unarchiver: adds support for 7z and other archive file types
  • ClamXav: free opensource Antivirus
  • Synergy: cross-platform opensource Keyboard and Pointer sharing
  • Virtualbox: For virtualization needs
  • Dropbox: For cloud storage
  • Skype: free video chat and messaging client
  • Transmission: opensouce torrent client
  • VLC: Video Player
  • Cyberduck: Network File Transfer client
Almost all the applications above can be installed using brew-cask except for Homebrew, Xcode and GGTS
brew install phinze/cask/brew-cask
brew cask install java
brew cask install mamp
brew cask install sublime-text
brew cask install netbeans
brew cask install brackets
brew cask install lighttable
brew cask install sourcetree
brew cask install sequel-pro
brew cask install google-chrome
brew cask install firefox
brew cask install opera
brew cask install trim-enabler
brew cask install iterm2
brew cask install alfred
brew cask alfred link
brew cask install the-unarchiver
brew cask install clamxav
brew cask install synergy
brew cask install virtualbox
brew cask install dropbox
brew cask install skype
brew cask install transmission
brew cask install vlc
brew cask install cyberduck

Monday, February 17, 2014

iPhone 3GS 6.1.3 Jailbreak Unlock

I recently got back to using my iPhone 3GS 16GB with the following specs:

Firmware: 5.1.1 Jailbroken Locked

Serial: 86011XXXXXX

Bootrom: New(fixed)

Baseband: 05.16.05

Some applications refused to run as they require a higher version of iOS so I decided to upgrade to 6.1.3.

Preparation

  • I downloaded the following two firmware packages:

  • Next I downloaded the latest version of RedSn0w. At the time of this writing: 0.9.15b3

Update to stock 6.1.3 using iTunes

  • I connected the iPhone 3GS to my windows PC with iTunes installed.
  • Allow iTunes to launch but skipped syncing and backup as I wanted a fresh/new phone
  • Turned off the phone via slide to poweroff
  • Switch to DFU mode by:

    • holding both power and home button for 10 seconds
    • then releasing the power button but continuing to hold the home button for 15 seconds
  • Restored stock 6.1.3 using iTunes by holding down the shift key will clicking the Restore button
  • Waited for phone to reboot and

Jailbreak and Unlock using RedSn0w

  • I connected the iPhone 3GS to my windows PC
  • Closed iTunes and Turned off the phone via slide to poweroff
  • Launched RedSn0w and clicked on Extra then Select IPSW
  • Selected the 6.0.0 IPSW file then went back and clicked on Jailbreak button
  • Selected Install Cydia and install ipad baseband
  • Followed onscreen instructions to complete jailbreak
  • Waited for phone to reboot
  • Relaunched RedSn0w and clicked on Extra then Select IPSW
  • Clicked on extra then Just Boot tethered button
  • Followed onscreen instrunctions to complete tethered boot
  • Waited for phone to reboot and launched Cydia
  • Searched Cydia for p0sixspwn and installed it to unthether
  • Relaunched RedSn0w and clicked on Extra then Select IPSW
  • Selected the 6.0.0 IPSW file then went back and clicked on Jailbreak button
  • Deselected Install Cydia and selected downgrade baseband
  • Followed onscreen instructions to complete baseband downgrade
  • Waited for phone to reboot and launched Cydia
  • Searched Cydia for ultrasn0w and installed it to unlock

Installed Jailbreak tweaks

  • Five Icon Dock
  • NoNewsIsGoodNews
  • Activator
  • SBSettings
  • biteSMS
  • RotationInhibitor

I'm beginning to wonder if jailbreaking is still worth it.

Monday, July 01, 2013

Ubuntu Server 12.04.2 KVM Host Setup

#!/bin/bash
#KVM Virtual machine host package installation:
apt-get install ubuntu-virt-server

#set scheduler on all sd? devices
find /dev/ -name sd? -type b | cut --delimiter='/' -f3 | while read -r; do echo "deadline" > /sys/block/$REPLY/queue/scheduler; done && \
find /dev/ -name sd? -type b | cut --delimiter='/' -f3 | while read -r; do echo -n "$REPLY:";cat /sys/block/$REPLY/queue/scheduler; done

#configure default io scheduler to deadline
sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="elevator=deadline"/g' /etc/default/grub && sudo update-grub

#configure fstab mount options to relatime(assumes ext4 file systems)
sed -i -r 's/ext4(\s+)errors/ext4\1relatime,errors/g' /etc/fstab

#remount all filesystems with relatime using UUID
cat /etc/fstab | grep relatime | cut -f1 --delimiter=' ' | cut -f2 --delimiter='=' | xargs --verbose mount -o remount -U

#install NTP and configure to sync time with at least 2 local NTP servers if available
apt-get install -y ntp && service ntp stop
read -p "Enter IP/URL of primary time server:" primary_ntp_server && read -p "Enter IP/URL of backup time server:" backup_ntp_server
cat <<EOF >/etc/ntp.conf
driftfile /var/lib/ntp/ntp.drift
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
server $primary_ntp_server iburst
server $backup_ntp_server
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery
restrict 127.0.0.1
restrict ::1
EOF
service ntp start

#configure bridge networking with eth0 (assumes eth0 has static ip configured)
sed -i -e '/^[[:space:]]*#/!s/eth0/br0/g' -e '/^[[:space:]]*[^#]*iface br0 inet/a \
\tbridge_ports eth0 \
\tbridge_fd 9 \
\tbridge_hello 2 \
\tbridge_maxage 12 \
\tbridge_stp off'  /etc/network/interfaces
invoke-rc.d networking restart

Ubuntu Server 12.04.2 KVM Guest Setup

preparing the template image:
#Setup io scheduler and serial console for virsh
sed -i -r 's/GRUB_CMDLINE_LINUX=".*"/GRUB_CMDLINE_LINUX="elevator=deadline console=ttyS0,38400n8 console=tty0"/g' /etc/default/grub && update-grub


#Setup serial terminal console config
cat <<EOF >/etc/init/ttyS0.conf
# ttyS0 - getty
#
# This service maintains a getty on ttyS0 from the point the system is
# started until it is shut down again.

start on stopped rc RUNLEVEL=[2345]
stop on runlevel [!2345]

respawn
exec /sbin/getty -L 38400 ttyS0 vt102
EOF


#Setup fstab mount options to relatime(assumes ext4 file systems)
sed -i -r 's/ext4(\s+)errors/ext4\1relatime,errors/g' /etc/fstab



#!/bin/bash
if [[ $EUID -ne 0 ]]; then
echo "script must be run as root. run with sudo" 2>&1
exit 1
fi

old_name=`hostname`
echo "Current hostname:$old_name"
echo -n "Enter new server name:"
read new_name
echo "$new_name">/etc/hostname
hostname $new_name
sed -i "s/$old_name/$new_name/g" /etc/hosts

#regenerate udev persistent net rules on reboot
echo "Removing /etc/udev/rules.d/70-persistent-net.rules"
rm /etc/udev/rules.d/70-persistent-net.rules

echo "Regenerating ssh server keys"
rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server

echo "*** You should probably change the password and reboot ***"

Monday, June 24, 2013

QT5 on xUbuntu 12.04.2 LTS

The latest version of xVideoServiceThief (2.5) switched to using the QT5 library.

To get it to work, I needed to setup QT5 on a xUbuntu 12.04.2 LTS which I did by running the following commands:

sudo apt-get install python-software-properties && \
sudo apt-add-repository ppa:canonical-qt5-edgers/qt5-proper && \
sudo apt-get update && \
sudo apt-get install libqt5gui5 libqt5webkit5 libqt5script5

Let me know in the comments if this helped you or if there were any issues.

Friday, June 21, 2013

Cloudstack 4.1 on Ubuntu Server

My objective was to setup a Cloudstack 4.1 management node which will also server an NFS server for both primary and secondary storage.

The series of commands below were run on an Ubuntu Server 12.04.2(precise) system. During installation of the operating system, only the OpenSSH server was enabled in the task selection screen. I also made sure to name the server with a fully qualified domain name i.e. cloudstack.henyo.com.

#setup cloudstack source
echo "deb http://cloudstack.apt-get.eu/ubuntu precise 4.1" > /etc/apt/sources.list.d/cloudstack.list
#setup percona source
cat <<EOF >/etc/apt/sources.list.d/percona.list
deb http://repo.percona.com/apt precise main
deb-src http://repo.percona.com/apt precise main
EOF

#install the cloudstack apt-key
wget -O - http://cloudstack.apt-get.eu/release.asc|apt-key add -

#install stuff
apt-get update && \
apt-get install -y --force-yes openntpd nfs-kernel-server cloudstack-management percona-server-server-5.5 percona-server-client-5.5

cat <<EOF >/etc/mysql/conf.d/cloudstack.cnf
[mysqld]
innodb_rollback_on_timeout=1
innodb_lock_wait_timeout=600
max_connections=350
log-bin=mysql-bin
binlog-format = 'ROW'
EOF
service mysql restart

#setup nfs
mkdir -p /export/primary /export/secondary /var/primary /var/secondary
cat <<EOF >>/etc/fstab
/var/primary    /export/primary   none    bind  0  0
/var/secondary  /export/secondary none    bind  0  0
EOF
mount --bind /var/primary/ /export/primary/
mount --bind /var/secondary/ /export/secondary/
echo "/export         *(rw,async,no_root_squash,no_subtree_check)" >> /etc/exports
exportfs -a
service nfs-kernel-server restart

#setup cloudstack database
cloudstack-setup-databases cloud:test@localhost --deploy-as=root:root

#setup management server
echo 'Defaults:cloud !requiretty' >> /etc/sudoers.d/cloudstack
cloudstack-setup-management

reboot


I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Friday, February 01, 2013

iphone 5 plan comparison

iphone 5 plan comparison from globe and smart


I took some time to dissect each iphone plan from both globe and smart.

The spreadsheet below compare the iphone 5 - 16GB plans on unlimited data.



The cheapest plan for the iphone 5 16GB comes from smart but the difference is really not that much especially if you look at the monthly cost. The smart plan also provides a bit more free all texts and minutes. To me, it still boils down to the network where most of your contacts are.



The iphone 5 16GB unit might not meet your storage needs so for an additional 200 pesos per month, you can avail of the iphone 5 32GB. Here, globe takes the lead by a measly 1 peso per month. But in my opinion, smart provides a lot more value in the form of freebies.

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Wednesday, January 23, 2013

Disabling BBU Auto Learn with megacli

We are currently facing performance problems with mysql and I remember reading about RAID BBU Learning causing huge write performance drops. So I wanted to check if the RAID controller on our Master database had this configured.

Step#1: Find out what is the brand/model of the RAID controller installed on the server

I can of course ask accounting to pull-up the delivery receipt to find the brand/model but where is the fun in that. Googling led me to the following commands:


sudo lspci | grep -i raid
04:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 2108 [Liberator] (rev 04)

sudo lshw -class storage
description: RAID bus controller
       product: MegaRAID SAS 2108 [Liberator]
       vendor: LSI Logic / Symbios Logic
       physical id: 0
       bus info: pci@0000:04:00.0
       logical name: scsi4
       version: 04
       width: 64 bits
       clock: 33MHz
       capabilities: storage pm pciexpress vpd msi msix bus_master cap_list rom
       configuration: driver=megaraid_sas latency=0
       resources: irq:26 ioport:d800(size=256) memory:fae7c000-fae7ffff memory:faec0000-faefffff memory:fae80000-faebffff

Step#2: Install the megacli package to be able to query the raid card for its status

Downloaded the megacli package from the LSI website. But they only provide RPMs so I had to convert them with:

sudo alien -k MegaCli-8.07.06-1.noarch.rpm

Then installed with:

sudo dpkg -i megacli_8.07.06-1_all.deb

To find out where the files got installed do:

sudo dpkg -c megacli_8.07.06-1_all.deb

Step#3: Use megacli to probe for BBU status and information

Running the command results to something unexpected:

./MegaCli64 -adpCount
Controller Count: 0.
Exit Code: 0x00

megacli cant find the raid adapter. It seems that megacli has an issue with kernels >= 3.0 and can be remedied with:

sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -adpCount
Controller Count: 1.
Exit Code: 0x01

So to find out about the BBU:

sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuStatus -aALL

BBU status for Adapter: 0

BatteryType: iBBU
Voltage: 3972 mV
Current: 0 mA
Temperature: 24 C
Battery State: Optimal
BBU Firmware Status:

  Charging Status              : None
  Voltage                                 : OK
  Temperature                             : OK
  Learn Cycle Requested                  : No
  Learn Cycle Active                      : No
  Learn Cycle Status                      : OK
  Learn Cycle Timeout                     : No
  I2c Errors Detected                     : No
  Battery Pack Missing                    : No
  Battery Replacement required            : No
  Remaining Capacity Low                  : No
  Periodic Learn Required                 : No
  Transparent Learn                       : No
  No space to cache offload               : No
  Pack is about to fail & should be replaced : No
  Cache Offload premium feature required  : No
  Module microcode update required        : No


GasGuageStatus:
  Fully Discharged        : No
  Fully Charged           : Yes
  Discharging             : Yes
  Initialized             : Yes
  Remaining Time Alarm    : No
  Discharge Terminated    : No
  Over Temperature        : No
  Charging Terminated     : No
  Over Charged            : No
  Relative State of Charge: 97 %
  Charger System State: 49168
  Charger System Ctrl: 0
  Charging current: 0 mA
  Absolute state of charge: 53 %
  Max Error: 2 %

Exit Code: 0x00

sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuCapacityInfo -aALL

BBU Capacity Info for Adapter: 0

  Relative State of Charge: 97 %
  Absolute State of charge: 53 %
  Remaining Capacity: 641 mAh
  Full Charge Capacity: 664 mAh
  Run time to empty: Battery is not being discharged.  
  Average time to empty: Battery is not being discharged.  
  Estimated Time to full recharge: Battery is not being charged.  
  Cycle Count: 37
Max Error = 2 %
Remaining Capacity Alarm = 120 mAh
Remining Time Alarm = 10 Min

Exit Code: 0x00

sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuProperties -aALL

BBU Properties for Adapter: 0
  Auto Learn Period: 30 Days
  Next Learn time: Sun Feb 17 19:37:09 2013
  Learn Delay Interval:0 Hours
  Auto-Learn Mode: Enabled
Exit Code: 0x00

Auto learn mode should be disabled and scheduled during off-peak time.

#!/bin/bash
TMPFILE=$(mktemp -p /tmp bbu.relearn.XXXXXXXXXX) || exit 1
echo "autoLearnMode=1" > $TMPFILE
setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -SetBbuProperties -f $TMPFILE -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -GetBbuProperties -aALL
rm $TMPFILE

script for safe write back:


#!/bin/bash
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp ADRA -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp -Cached -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp DisDskCache -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp NoCachedBadBBU -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp WB -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aALL


script to force write back without BBU protection:

#!/bin/bash
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp ADRA -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp -Cached -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp DisDskCache -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp CachedBadBBU -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDSetProp WB -Lall -aALL
sudo setarch x86_64 --uname-2.6 /opt/MegaRAID/MegaCli/MegaCli64 -LDInfo -Lall -aALL


Useful Links:
http://linux.dell.com/files/whitepapers/solaris/Managing_PERC6_0714.pdf
http://hwraid.le-vert.net/wiki/LSIMegaRAIDSAS
http://yo61.com/dell-drac-bbu-auto-learn-tests-kill-disk-performance.html


I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Wednesday, November 21, 2012

Transferring a huge file over a LAN

We need to transfer several huge files from an old server to a new server. I plan to use netcat to do a raw transfer followed by rsync to ensure data integrity. I'll report back on the results.


On the destination server:
nc -l 1234 > ubuntu-12.04.1-server-i386.iso


On the source server:
time cat ubuntu-12.04.1-server-i386.iso | nc $DESTINATION_IP 1234


On the source server:
cat >rsyncd.conf <<EOF
#rsyncd.conf
[temp]
path=/var/lib/libvirt/images/
read only = yes
list = yes
use chroot = no
EOF
rsync --config=rsyncd.conf --daemon --no-detach --port=1234


On the destination server:
time rsync -av --inplace --progress rsync://apollo@192.168.7.167:1234/temp/ubuntu-12.04.1-server-i386.iso ubuntu-12.04.1-server-i386.iso

Update#1:

time cat ubuntu-12.04.1-server-i386.iso | nc 192.168.7.198 1234

real 0m58.130s
user 0m0.072s
sys 0m2.016s

ubuntu-12.04.1-server-i386.iso is 646MB in size which gives a transfer rate of 11.13MB/s which is NOT what I expected from a gigabit connection.



I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.


Monday, April 30, 2012

Safely Rebooting a Hanging Server


When all else fails, go to the physical server console and hit:
Alt-SysRq R (wait for 21 secs)
Alt-SysRq E (wait for 21 secs)
Alt-SysRq I (wait for 21 secs)
Alt-SysRq S (wait for 21 secs)
Alt-SysRq U (wait for 21 secs)
Alt-SysRq B

Then cross your fingers as the server should reboot

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Friday, April 13, 2012

Problem with grails, maven and spring security

Environment:
Ubuntu 12.04 3.2.0-20-generic-pae kernel
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.1) (6b24-1.11.1-3ubuntu3)
OpenJDK Server VM (build 20.0-b12, mixed mode)

I installed STS 2.9.1 RELEASE which includes apache-maven-3.0.3

I placed the apache-maven-3.0.3 in my PATH environment variable

I created a project with:
mvn archetype:generate \
-DarchetypeGroupId=org.grails \
-DarchetypeArtifactId=grails-maven-archetype \
-DarchetypeVersion=1.3.7 \
-Dversion=1.0.0alpha \
-DgroupId=com.henyo.foobar -DartifactId=foobar

mvn proceeded to download the internet and then created the project folder with the pom file
I modified the pom file to set the source and target to 1.6 as described in the grails manual

I then did:
mvn initialize
which created the grails directory structure

I tried:
mvn grails:run-app
which resulted in the app successfully running

I then tried running just the unit tests with:
mvn grails:exec -Dcommand=test-app -Dargs="--unit"
which resulted in successful build

I then installed the spring security core plugin with:
mvn grails:install-plugin -DpluginName=spring-securit-core

I then tried using the s2-quickstart script:
mvn grails:exec -Dcommand=s2-quickstart -Dargs="com.henyo.foobar.model User Role"
which resulted in a build FAILURE:
[ERROR] Failed to execute goal org.grails:grails-maven-plugin:1.3.7:exec (default-cli) on project foobar: Unable to start Grails: java.lang.reflect.InvocationTargetException: org/springframework/security/core/Authentication: org.springframework.security.core.Authentication -> [Help 1]

It seems that the grails maven integration still needs a lot of work. I suspect this is a dependency resolution issue.

Uploaded a sample project that has this issue: http://dl.dropbox.com/u/5541070/foobar.tar.gz

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Books for Software Development


My updated reading list

Learning Programming
-------------
How to think like a computer scientist: Python Edition
Thinking in C beta 3
The Passionate Programmer


Learning Java
-------------
Head First Java 2nd Ed
How to think like a computer scientist: Java Edition
Objects First with Java
Thinking in Java 4th Ed
Big Java
Agile Java: Crafting Code with Test-Driven Development
Core Java Vol 1 & 2
The Art & Science of Java
The Java Programming Language

Programming Practices
---------------------
The Pragmatic Programmer: From Journeyman to Master
Pragmatic Version control using git
Pragmatic Unit testing in Java with JUnit
Pragmatic Project Automation
Debug It
Ship it
Release it

Learning Basic Web Frontend Development
---------------------------------------
Head First HTML with CSS & XHTML
Head First Javascript
Javascript the Good Parts
Stylin' with CSS: A Designer's Guide (2nd Edition)
Designing with Web Standards by Jeffrey Zeldman and Ethan Marcotte
Web Design for Developers: A Programmer's Guide to Design Tools and Techniques by Brian P. Hogan
Eric Meyer on CSS: Mastering the Language of Web Design by Eric Meyer
JQuery in Action by Bear Bibeault and Yehuda Katz
Handcrafted CSS by Dan Cederholm with Ethan Marcotte

Learning Swing Frontend Development
-----------------------------------
Desktop Java Live
Swing Second Edition
Java Swing

Mastering Software Development
------------------------------
Head First Object-Oriented Analysis and Design
Head First Design Patterns
Code Complete: A practical handbook of Software Construction
Clean Code: A Handbook of Agile Software Craftsmanship
Program Development in Java
The Elements of Java(TM) Style
Effective Java
Practical Java
Design Patterns: Elements of Reusable Object-Oriented Software
Patterns of Enterprise Application Architecture
Domain-Driven Design
Refactoring: Improving the Design of Existing Code
Refactoring to Patterns
SQL Anti-patterns
Modular Java
Growing Object-Oriented Software Guided by Tests
Holub on Patterns:Learning Design Patterns by Looking at Code
Test Driven Development by Kent Beck
XUnit Test Patterns - Refactoring Test Code by Gerard Meszaros
Concurrency Practice in Java
Java Generics and Collections
Java Performance
Java Puzzlers

Methodology and Management
--------------------------
UML Distilled, 2nd Edition
Extreme Programming Explained
The Unified Software Development Process
Rapid Development: Taming Wild Software Schedules
Peopleware: Productive Projects and Teams
The Mythical Man-Month
Software Runaways: Monumental Software Disasters
Software Creativity
Agile Estimating and Planning by Mike Cohn
Agile Software Development, Principles, Patterns, and Practices by Robert (Bob) C. Martin
Agile Retrospectives: Making Good Teams Great by Esther Derby, Diana Larsen, and Ken Schwaber
Succeeding with Agile: Software Development Using Scrum by Mike Cohn
Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory
Extreme Programming Installed by Ron Jeffries, Ann Anderson, and Chet Hendrickson
Behind Closed Doors: Secrets of Great Management by Johanna Rothman, and Esther Derby
Leading Lean Software Development: Results Are Not the Point by Mary and Tom Poppendieck
Lean-Agile Software Development: Achieving Enterprise Agility by Alan Shalloway, Guy Beaver, and James R. Trott
The Art of Agile Development by James Shore and Shane Warden

Mastering Groovy and Grails
---------------------------
Groovy in Action
The Definitive Guide to Grails
Grails in Action
Groovy Recipes
Programming Groovy

Mastering Web Design
--------------------
Web Design for Developers
Head First Web Design
Designing Web Usability

Mastering User Interface Design
-------------------------------
User Interface Design for Programmers
Don't Make me Think
About Face: The Essentials of User Interface Design
Design of Everyday Things
The Non-Designer's Design Book

Mastering Hardcore Stuff
------------------------
Code: The Hidden Language of Computer Hardware and Software by Charles Petzold
The C Programming Language
Structure and Interpretation of Computer Programs
Introduction to Algorithms by CLRS (Cormen,Leiserson,Rivest,Stein)
Compilers: Principles, Techniques and Tools
Test Driven Development for Embedded C
The Art of Computer Programming


- Head First Java
- Head First Object Oriented Analysis and Design
- Head First Design Patterns
- Effective Java
- Concurrency Practice in Java
- Java Generics and Collections
- Java Performance
- Java Puzzlers
- Thinking in Java
- Clean Code


You are also welcome to subscribe via email or subscribe via reader to get updates.

Tuesday, April 03, 2012

Trying Ubuntu 12.04LTS Precise Pangolin

I migrated my desktop from Ubuntu 10.04LTS to Ubuntu 12.04LTS.

A few things I needed to add after installation:
  • Chromium as my default browser
  • adobe-flash to be able to play flash games on facebook
  • openjdk-6-jdk for work stuff
  • icedtea-plugin to be able to run charting applet on citisec
  • pidgin for messaging
  • git for more work
  • smartgit for routine pushing and comitting to git repos
  • smartsvn for routine pushing and comitting to git repos
  • STS for work stuff
  • DB Visualizer for work stuff
  • Virtual Box for desktop virtualization
  • Dia for diagramming
  • synergy to share one keyboard between my desktop and laptop
  • dropbox for personal file sharing
  • pencil for mockups
  • smplayer for watching videos
  • vlc for videos that do not work with smplayer
  • git gui for simple git commits
  • myqsl-server for development work
  • mysql workbench for managing mysql database
  • postgresql for development work
  • pgadmin for managing postgres database

Should I be replacing stuff on the list with something else? What other stuff have I forgotten to install?

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Monday, April 02, 2012

Protecting ProxmoxVE VMs with IPCop on KVM

I found out a few weeks ago that protecting openvz virtual machines running ubuntu was a bit problematic as I was unable to run UFW on them.

I was able to setup IPCop v2.04 on KVM which will act as a virtual firewall for all of the virtual machines in our ProxmoxVE 2.0 cluster

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Thursday, March 29, 2012

Building a good Software Startup

If I were to build a good software startup today, I'd run with the following ideas.

Hire only good people who get things DONE.
Even if they are more expensive in terms of compensation.
Quality trumps numbers. No doubt about it.


Development Team
  • 1 Lead Developer
  • 1 Senior Developer
  • 2 Junior Developers
  • 2 Business Analyst/QA

Operations Team
  • 1 Lead Systems Admin
  • 1 Senior System Admin
  • 6 Junior System Admins

Give your team a quiet working environment with ergonomic furniture and equipment

Give your team good internet connection
  • the faster they can download their installers, the earlier they can start working on code
  • the faster they can look up the solution on google or stackoverflow, the faster they can proceed to the next problem

Give your team good computer hardware
  • at least dual core processors with virtualization support
  • at least 8GB memory
  • at least 2 hard drives for mirror setup
  • at least 2 screens

Make sure to have enough environments to test, deploy and support the system properly.
  • Development Environment
  • Staging Environment
  • Support Environment
  • Production Environment

Make use of Free or Open Source systems to manage the development process.
Buy a commercial product only if you can't find a solution that fits.


Project/Issue Tracking with Source Code Management Options:
  • Indefero
  • Gitorious
  • Gitlab
  • Redmine+gitolite

Continuous Integration Server Options:
  • Jenkins
  • Team City

Code Quality:
  • Sonar

Setup google apps for for email, calendar and other stuff

As an example, here is how I would set it up on three physical servers:

Server#1 running ProxmoxVE
- Firewall/Proxy Server/Package Cache(apt,yum,mvn) VM
- Redmine+gitolite VM
- Jenkins VM
- Sonar VM
- NFS/Samba Server VM

Server#2 running ProxmoxVE
- Development Environment VM
- Staging Environment VM
- Support Environment VM

Server#3 for storage of backups
- nexentastor or freenas or centos+zfsonlinux

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Wednesday, March 28, 2012

ProxmoxVE 2: Configuring DRBD

I am running ProxmoxVE 2.0-35/d07f49c3 on two nodes connected using a cross cable on gigabit nics.
host1 ip: 10.0.10.201
host2 ip: 10.0.10.202

Check if drbd module is available
modprobe -l | grep drbd
Check if drbd module is loaded
lsmod | grep drbd
If not, load it with
modprobe -v drbd
Check the module version
modinfo drbd | grep version
I installed the appropriate drbd8-utils package on both nodes
dpkg -i drbd8-utils_8.3.10-0_amd64.deb
I built my own deb from source. You can find my build here.

I shrank the data volume on both of
I made use of the extents I freed up by shrinking the data volume to create an lvm volume that will be used for drbd.
lvcreate --verbose --extents 50000 --name lv4drbd pve
Next I modified /etc/drbd.d/global_common.conf so that it only contains:
global {
    usage-count no;
}
common {
    protocol C;
    syncer {
        rate 30M; 
    }
}
Next I created a new resouce file /etc/drbd.d/drbd0.res
resource drbd0 {
    protocol C;
    startup {
        wfc-timeout  0;
        degr-wfc-timeout 60;
        become-primary-on both;
    }
    net {
        cram-hmac-alg sha1;
        shared-secret "drbd0-secret";
        allow-two-primaries;
        after-sb-0pri discard-zero-changes;
        after-sb-1pri discard-secondary;
        after-sb-2pri disconnect;
    }
    on host1 {
        device /dev/drbd0;
        disk /dev/pve/lv4drbd;
        address 10.0.10.201:7780;
        meta-disk internal;
    }
    on host2 {
        device /dev/drbd0;
        disk /dev/pve/lv4drbd;
        address 10.0.10.202:7780;
        meta-disk internal;
    }
}
I added the IP address of the other host into each of the /etc/hosts files.

Next I started drbd on both nodes with
/etc/init.d/drbd start
I then initialized drbd metatdata on both nodes with
drbdadm create-md drbd0
Next I brought up the device on both nodes with
drbdadm up drbd0
I checked the status on both nodes with
drbd-overview
Next I started synchronization(should be instantaneous) from one node with
drbdadm -- --clear-bitmap new-current-uuid drbd0
Lastly, I restarted the drbd service on both nodes to enable Primary/Primary operation
/etc/init.d/drbd stop
/etc/init.d/drbd start

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

ProxmoxVE 2.0: Shrinking the data volume

After installing ProxmoxVE 2.0rc1, a data volume was created which makes use of most of the space on the drive.

I wanted to carve out a separate volume for some other use so I needed to shrink this data volume:

First, I unmounted the logical volume with:
umount /var/lib/vz
Then I proceeded to shrink the file system to the minimum size possible with:
resize2fs -Mf /dev/pve/data
Then I proceeded to reduce the size of the logical volume with:
lvreduce --verbose --extents -100000 /dev/pve/data
Then I proceeded to grow the file system back to occupy the now reduced volume with:
resize2fs -f /dev/pve/data
Lastly, I remounted the file system with:
mount /dev/pve/data /var/lib/vz

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Monday, March 26, 2012

FizzBuzz in Javascript

Joel and company probably won't hire me since he only gets rockstar developers but here is my attempt at FizzBuzz
<script>
    for(var i=1;i<=100;i++){
        if(i%3 != 0  && i%5 != 0)
            document.write(i);
        if(i%3 == 0)
            document.write('Fizz');            
        if(i%5 == 0)
            document.write('Buzz');                    
        document.write(' ');
    } //end for loop
</script>
I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Friday, March 23, 2012

dailyLooper bash script


We needed to regenerate some data for the data warehouse so I had to run an extraction script once for each day of the month.

I wrote a simple bash script to make it easier to run a command or other scripts once for every day of a target month and year.

Code can be found below but may also be downloaded from here.

#!/bin/bash
usage(){
    echo "Usage: dailyLooper <year> <month> <script to run>"
    echo "Each date in the given month will be passed into the script in yyyy-mm-dd format"
}


CURRENT_MONTH=$(date +%m)
TARGET_YEAR=$1
TARGET_MONTH=$2

if [ "$#" -lt 3 ]; then
    usage
    exit 1
fi

if [ "$3" = "" ]; then
    SCRIPT="echo"
else
    SCRIPT=$3    
fi

#returns the date yesterday if target month is the current month
#otherwise returns the last date for the target month
lastDayOfMonth(){
    local _year=$1
    local _month=$2   
    if [ $_month -lt $CURRENT_MONTH ]; then
        echo $(date -d "${TARGET_YEAR}-${TARGET_MONTH}-01 + 1 month - 1 day" +%d)
    else
        echo $(date -d "yesterday" +%d)
    fi    
}


main(){
    local _lastDay=$(lastDayOfMonth $TARGET_YEAR $TARGET_MONTH)
    i=1
    while [ "$i" -le $_lastDay ]
    do
        local _date=$(date -d "${TARGET_YEAR}-${TARGET_MONTH}-${i}" +%Y-%m-%d)
        $SCRIPT $_date
        ((i++))
    done #end day of month loop
}

main

I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Tuesday, March 20, 2012

Cheap and Safe File Storage on Linux


I needed to setup a Samba file server using off the shelf desktop components.
It will be used in a small office with 5-10 clients.

Here are the relevant hardware specs:
Processor: i7
Memory: 16GB
Drives: 4 x 1TB sata drives
OS Installed: Ubuntu Sever 10.04LTS

The primary requirement is for the files to be fairly safe i.e. if the server crashes in the middle of saving a file then on restart, the file should return to the consistent state before the save operation. Performance is secondary to safety.

My initial plan was to use one drive for the OS and configure 3 drives in an mdraid level 5. After some research, I came across several references that state that mdraid level 10 works on odd numbered drives. It will work on 3 drives and even on 2. This looked like something I should try. I then had to decide what filesystem to use. I initially wanted to use xfs as I've heard and read a lot of good things about it.

After studying how to setup both mdraid and xfs, I started a thread on the linux-raid mailing list asking for best practice guidelines and guidance on using xfs with mdraid 10n2.

Several days of additional research guided by the comments on the mailing list thread resulted in a lot of learnings for me:
  • raid10n2 really works on 3 drives, I had my doubts at first but now I finally get how it can work with just three drives
  • drive sector size can be either 512b or 4k which can lead to mis-aligned partitions and have a negative impact on performance
  • ext4 provides the option: data=journal which makes it safer than xfs at the cost of some performance

The table below shows the average of the benchmark results I got. Yes, I know bonnie++ only tests sequential write/rewrite/read and some file operations and is not necessarily representative of the workload but IMHO, it is better than nothing. This was how things were setup for the benchmark:
  • a 50GB primary partition was created on each device located 1GB from the start of the disk for partition alignment
  • the md raid device is created on top of the primary partition created on each device
  • the chunk was set to 64k
  • the results below are for raid5 and raid10n2 only but I also tried raid10f2 and raid10n2
  • raid 10f2 had better read performance but slower write compared to raid 10n2
  • raid 10o2 performed quite closely to raid 10n2
  • the benchmark was run three times and the average was taken

Bonnie 1.96 Sequential Output Sequential Input Random

Seeks
64k chunk Size Block Rewrite Block
ext4/mdraid
K/sec % CPU K/sec % CPU K/sec % CPU /sec % CPU
raid5
data=ordered
32G 144430 11 68725 9.67 249364 21 280.33 9.33
raid5
data=journal
32G 25077 4.67 21605 5 267283 22.33 293.67 6
raid10n2
data=ordered
32G 137365 8 69940 9 209180 16 365.03 11.67
raid10n2
data=journal
32G 56249 6 37491 6 206548 16 389.00 8

For those interested, you can find/get the script I used here.

I modified this script to run my tests.

You can find the raw results here.

You can also find the debug info here.

I am considering re-running the tests but this time with iozone instead of bonnie++ so that I can get scores for random read and write.

Next, I plan to do pull-the-plug testing on ext4,data=journal on top of raid10n2 while doing streaming writes of a huge file over samba. I plan to use 2 different video files greater than 3GB. First I will copy file1 to the samba share and time it. I will then compute a checksum for file1 both locally and on the samba server. The computed checksum should be equal. Next, I will compute a checksum for file2 locally. It should be different from the value computed for file1. I will then move file1 to another directory and rename file2 to the the same name as file1. I will then attempt to copy the new file1 to the samba server. Halfway into the copy, I will pull the plug of the samba server. Next I will restart the samba server and recompute the checksum of file1. It should still be equal to the computed copy before.

If the system behaves as expected, I will go with ext4,data=journal on top of raid10n2.


I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.

Wednesday, March 14, 2012

Check disk write-caching


I'm investigating the performance of file systems and found myself needing to check if write-caching was enabled for my disk.
hdparm -W /dev/sda
To turn off write-caching, do
hdparm -W0 /dev/sda


I hope you found the post useful. You can subscribe via email or subscribe via a feed reader to get relevant updates from this blog. Have a nice day.