Mozy for the headless Mac 0

Posted by jro
on Saturday, March 15

Frequently mentioned on Lifehacker, Mozy Home is an incredibly economical network backup option at $5 a month. The online backup service will store an unlimited amount of data, and has Mac and Windows clients. The big caveat there is that you don’t have unlimited throughput, so it can still take a while to backup a large fileserver, say. In that case it’s especially important to be able to manage it effectively.

I’m currently using an old PPC-based Mac Mini, running Tiger, with a couple (or more) USB attached drives, as my home fileserver. For a long time I ran a Linux box with an internal RAID5 disk array, but with the advent of mozy and 5$ a month offsite backup, I’ve made the switch. It hasn’t been completely painless, however, so here are a couple notes about effectively using Mozy on a headless Mac.

  • You still probably need VNC or that fancy new Leopard desktop-sharing feature, especially to install.

  • Mozy has a command line interface! Enable SSH on your Mac and check this out

bombe:~ jro$ /Applications/Mozy.app/Contents/Resources/MozyBackup help
Commands: start|cancel|auth|rescan|status|tempdir

bombe:~ jro$ /Applications/Mozy.app/Contents/Resources/MozyBackup status
Status: Backing up files...

Preparing (0.0 B/s): 
   70.9% 4925415586 of 6943326336

 Sending   (46.3 KB/s): blah_blah_20061012.tgz
    0.9% 87851008 of 9257768960

You can probably guess what most of the commands do. It’s great being able to start and cancel a backup on a whim, say when I upload more data to the server, or stop it when it’s messing with my VOIP call quality. I definitely recommend still having Mozy backup on a regular schedule, even if you’re doing a fair bit by hand. You never know if you’ll forget!

  • Mozy has a very useful log.
bombe:~ jro$ sudo less /Library/Logs/Mozy.log

2008-03-15 12:43:55.229 MozyBackup[31] (backup) Starting backup
2008-03-15 12:43:55.231 MozyBackup[31] (backup) Checking for update
2008-03-15 12:43:56.821 MozyBackup[31] (backup) Connecting to r4165.mozy.client.ut2.berkeleydata.com (66.133.112.25)
2008-03-15 12:43:57.277 MozyBackup[31] (backup) Version is up-to-date (0.9.1.0)
...
2008-03-15 12:46:01.028 MozyBackup[31] (backup) Building backup set
2008-03-15 12:46:03.278 MozyBackup[31] (backup) Processing files in scan cache
2008-03-15 12:46:09.704 MozyBackup[31] (backup) Processing files in manifest but not in scan cache
2008-03-15 12:46:19.569 MozyBackup[31] (backup) Preparing 4 files (6943326336 bytes)
2008-03-15 12:46:19.570 MozyBackup[31] (backup) Unlocking cache...
2008-03-15 12:46:19.570 MozyBackup[31] (backup) Unlocked cache
2008-03-15 12:46:19.570 MozyBackup[31] (prepare) Starting
2008-03-15 12:46:19.575 MozyBackup[31] (send) Starting
2008-03-15 12:46:19.575 MozyBackup[31] (send) Getting files from queue
2008-03-15 12:51:12.245 MozyBackup[31] (send) Offering 1 files (0 are removes)
  • Mozy can use your your encryption key. If you don’t want the guys at EMC/Mozy to know what kind of ahem data you’re storing on their machines, you can configure Mozy to use your own encryption key. This definitely increases the security of your data, just make sure you don’t lose that key, or you’ll be in a world of hurt come restore time.

fink error: environment variable set to: 10.1 2

Posted by jro
on Monday, January 21

I, recently, decided to move some of my home rails sites to my aging PPC-based mac mini. I’m still running Tiger on it because I’m lazy, don’t need any new features, and know how macports & fink work in Tiger. I ran through the normal, install fink, ruby package,… routine. But, when I got to compiling some of my gems, I got this error:

/usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/ld: flag: -undefined dynamic_lookup can't be used with MACOSX_DEPLOYMENT_TARGET environment variable set to: 10.1

Which seems pretty strange since I’m obviously running on 10.4. And, while I don’t know why the system is getting it wrong, a quick work around is to set the environment variable yourself.

export MACOSX_DEPLOYMENT_TARGET=10.4

Oddly, it doesn’t happen on my intel macs, so I suspect that there’s a bug somewhere in the PPC fink stack that just doesn’t effect all compilations. /shrug.

Packaging Software: Introduction to RPM 3

Posted by jro
on Sunday, December 02

Better late than never. This is the first “real” article in the Packaging Software series. It’s long, but hopefully it’ll help someone out. Enjoy!

So what’s an RPM?

In it’s distilled form, a big ball of files that hopefully make up something useful, like software. But wait, there’s more! It’s not just a file format for installing, uninstalling, updating, verifying software, it’s also the system for creating those bundles and doing all the installing, etc. Confused? What we have is two parts: The files commonly referred to as RPM, aka the software bundles, and rpm set of commands for creating and managing those bundles.

History and the Soap Opera

RPM was birthed by Red Hat as the Red Hat Package Manager, later becoming the RPM Package Manager as it became building block of more and more non-Red Hat Linux distributions. It’s grown to be used in innumerable Linux distros, large and small, popular and not.

I don’t want to get the drama wrong, so I’m just going to quote Wikipedia

As of the 31st of May, 2007, there are two versions of RPM in development. The first is led by long-time (since 1999) RPM maintainer, Jeff Johnson (‘JBJ’), who continued his development efforts after departing Red Hat. It has combined recently with efforts of OpenPKG and added a new website rpm5 tine of the fork. The ‘JBJ’ variant is used by Mandriva, PLD Linux Distribution and supported by OpenPKG. Jeff Johnson formerly worked for Red Hat before a controversy.

The other effort is led by Red Hat recent hire (2007) staff member Panu Matilainen. The Red Hat RPM tine of the fork which issued its first major code revision in July 2007, is used by Fedora, Red Hat Enterprise Linux, Novell’s openSUSE and CentOS. Matilainen has also worked on an apt-like front end for package management for RPM previously; apt-get is a Debian Linux management tool.

The displaced former RPM website content is archived by its long time maintainer at oldrpm web archive. On December 14, 2006, Red Hat decided to take complete control of editorial content at the formerly community maintained website which content was maintainted in an ‘open to the community’ process manner.

Does it matter to you? Probably not, just be aware there are currently two major forks of RPM and there may be slight differences in the API standards or tools that are available.

The simple things

So let’s do something useful. I’m working on a Red Hat system, so if you’re not it’s possible things will be a little different.

What do we have installed?

List them all

Good for grep’ing through when you don’t know what you’re looking for.

[jro@localhost ~]$ rpm -qa 
libgcc-3.4.6-8.i386
filesystem-2.3.0-1.i386
glibc-common-2.3.4-2.36.i386
bzip2-libs-1.0.2-13.EL4.3.i386
...
Find a specific package

Good when you know it’s there, but just need the version.

[jro@localhost ~]$ rpm -q rmt
rmt-0.4b39-3.EL4.2.i386
Find out what RPM owns a file

Good for.. where the hell did that come from?

[jro@localhost ~]$ rpm -qf /etc/snmp/snmpd.conf
net-snmp-5.1.2-11.EL4.10.i386
Get some basic info on an installed package.
[jro@localhost ~]$ rpm -qi hdparm
Name        : hdparm                       Relocations: (not relocatable)
Version     : 5.7                               Vendor: Red Hat, Inc.
Release     : 2                             Build Date: Tue 21 Sep 2004 05:23:51 AM CDT
Install Date: Wed 27 Jun 2007 04:10:31 PM CDT      Build Host: tweety.build.redhat.com
Group       : Applications/System           Source RPM: hdparm-5.7-2.src.rpm
Size        : 64075                            License: BSD
Signature   : DSA/SHA1, Wed 05 Jan 2005 05:18:55 PM CST, Key ID 219180cddb42a60e
Packager    : Red Hat, Inc. 
Summary     : A utility for displaying and/or setting hard disk parameters.
Description :
Hdparm is a useful system utility for setting (E)IDE hard drive
parameters. For example, hdparm can be used to tweak hard drive
performance and to spin down hard drives for power conservation.
List all the files installed as part of a package

Good for where the hell did that config file get put?

[jro@localhost ~]$ rpm -ql rmt
/etc/rmt
/sbin/rmt
/usr/share/man/man8/rmt.8.gz

Installing and Upgrading RPMs

Installing a single RPM
[jro@localhost ~]$ rpm -i nginx-0.5.31-1.vp.i386.rpm
error: can't create transaction lock on /var/lock/rpm/transaction

Hrmm.. maybe I need to do this as root

[jro@localhost ~]$ sudo rpm -i nginx-0.5.31-1.vp.i386.rpm
Password: *****
[jro@localhost ~]$

Yep. Not a lot of output tho

[jro@localhost ~]$ sudo rpm -iv nginx-0.5.31-0.vp.i386.rpm
Preparing packages for installation...
nginx-0.5.31-0.vp

Better.

[jro@localhost ~]$ sudo rpm -ivh nginx-0.5.31-0.vp.i386.rpm
Preparing...                ########################################### [100%]
   1:nginx                  ########################################### [100%]

There we go.

Installing multiple packages at the same time
[jro@localhost ~]$ sudo rpm -ivh nginx-contrib-0.5.31-0.vp.i386.rpm
error: Failed dependencies:
        nginx = 0.5.31 is needed by nginx-contrib-0.5.31-0.vp.i386
[jro@localhost ~]$ sudo rpm -ivh nginx-0.5.31-0.vp.i386.rpm
Preparing...                ########################################### [100%]
   1:nginx                  ########################################### [100%]
[jro@localhost ~]$ sudo rpm -ivh nginx-contrib-0.5.31-0.vp.i386.rpm
Preparing...                ########################################### [100%]
   1:nginx-contrib          ########################################### [100%]

So our first package required another, and installing them one by one worked well enough but it’s a pain.

[jro@localhost ~]$ sudo rpm -ivh nginx-contrib-0.5.31-0.vp.i386.rpm nginx-0.5.31-0.vp.i386.rpm
Preparing...                ########################################### [100%]
   1:nginx                  ########################################### [ 50%]
   2:nginx-contrib          ########################################### [100%]

It’s way easier just to put them all together on one line. Or go c-razy and use file globbing ( rpm -ivh *.rpm ).

Upgrading RPMs
[jro@localhost ~]$ sudo rpm -Uvh nginx-contrib-0.5.31-1.vp.i386.rpm nginx-0.5.31-1.vp.i386.rpm
Preparing...                ########################################### [100%]
   1:nginx                  ########################################### [ 50%]
   2:nginx-contrib          ########################################### [100%]

It’s not too much different than installing, just that the -i turns into a -U. HOWEVER, you can also use the -U to install a package. So why bother using -i? It’s simple. Don’t. Always use -U, unless you specifically don’t want to upgrade a package you might have installed. It sounds silly, but you might end up in that situation one day if you have an RPM that restarts a service, etc.

Uninstalling.. er Erasing RPMs

[jro@localhost ~]$ sudo rpm -e nginx
Password:
error: Failed dependencies:
        nginx = 0.5.31 is needed by (installed) nginx-contrib-0.5.31-1.vp.i386
[jro@localhost ~]$ sudo rpm -e nginx-contrib nginx
[jro@localhost ~]$                                       

Pretty much what you’d expect at this point.

Useful Magic

See the architecture of a package

If you’re running a 64-bit machine, it’s VERY useful to know which packages are 32-bit and which are 64-bit. As on some distros it’s possible to have both versions installed, and then things start to get very confusing.

[jro@localhost ~]$ rpm -qa | head -5
libgcc-3.4.6-8
filesystem-2.3.0-1
glibc-common-2.3.4-2.36
bzip2-libs-1.0.2-13.EL4.3
ethtool-1.8-4
[jro@localhost ~]$ echo "%_query_all_fmt         %%{name}-%%{version}-%%{release}.%%{arch}" >> ~/.rpmmacros
[jro@localhost ~]$ rpm -qa | head -5
libgcc-3.4.6-8.i386
filesystem-2.3.0-1.i386
glibc-common-2.3.4-2.36.i386
bzip2-libs-1.0.2-13.EL4.3.i386
ethtool-1.8-4.i386

We’ll talk more about the rpmmacros files when we get to building packages, but I recommend adding this even if you’re not running a system with packages from differing architectures.

See the changelog for a package

Installed package
[jro@localhost ~]$ rpm -q --changelog nginx 
* Wed Sep 19 2007  0.5.31-1.vp

- add condrestart

* Wed Sep 19 2007  0.5.31-0.vp

- upgrade to latest stable

* Mon Sep 17 2007  0.5.26-3.vp
....
Package not yet installed
[jro@localhost ~]$ rpm -qp nginx-0.5.26-0.vp.i386.rpm --changelog
* Mon Jul 02 2007  0.5.26-0.vp

- new nginx version - 0.5.26
- start fork of package

* Wed May 23 2007  0.5.20-0

- new nginx version - 0.5.20

* Thu Apr 26 2007  0.5.19-0

Muck with the output format of RPM without making it semi-permanent

[jro@localhost ~]$ rpm -qa --queryformat "%{NAME}\n" | head -3
libgcc
filesystem
glibc-common
[jro@localhost ~]$ rpm -qa --queryformat "%{NAME}_%{VERSION}\n" | head -3
libgcc_3.4.6
filesystem_2.3.0
glibc-common_2.3.4
[jro@localhost ~]$ rpm -qa --queryformat "%{NAME}_%{VERSION}_%{DISTRIBUTION}\n" | head -3
libgcc_3.4.6_Red Hat (RHEL-4)
filesystem_2.3.0_Red Hat Linux
glibc-common_2.3.4_Red Hat (RHEL-4)

It’s not super intuitive, but thankfully there’s some info in the rpm man page, and it can be handy when writing scripts to report on your packages, create lists to mirror systems from, etc.

Resources

  • The RPM Guide at The Fedora Project - This used to be a book that’s no longer in print. This is the best resource outside of looking at what other people have done.
  • The RPM HOWTO at The Linux Documentation Project - It’s a bit dated but there’s still some relevant information there.
  • RPM.org wiki - Links to other resources.

nginx for fink 2

Posted by jro
on Wednesday, November 14

I just submitted an nginx package to fink after waiting about a month to define which files were documentation. Hopefully, it’ll be validated and added quickly, but if you need it now head over to sourceforge to grab the .info and .patch. The packaging tutorial over at finkproject should have all the info you’d need to build it for yourself. As well as teach you how simple it is to get started working on packages for fink.

Tales from the server: Packaging Software 0

Posted by jro
on Sunday, September 30

You’ll often hear SysAdmins advocate writing a script for any task you expect to repeat more than once, and I couldn’t agree more. Over the years, I’ve often found myself taking it a step further, and packaging anything I expect to install more than once, or install in any production environment. There are a lot of benefits, but mostly I find the added server consistency and conveniences of the package management tools make up for any extra effort required in bundling it together.

Of course you could do the same thing with your own custom scripts, but I don’t like reinventing the wheel and I really like the extra integration of using the same system as my OS. So what makes a good packaging system? Here’s my list:

  • Versioning
  • Robust dependencies
  • Able to upgrade and downgrade packages
  • Pre-install, Post-install scripting
  • Binary packages - my preference, definitely not a hard requirement

With that in mind, consider this the kick-off to a series on effectively building and mixing your own packages with your OS distribution of choice. I’ll be focusing on RPMs as the package type on RedHat / Fedora Linux distributions, but the patterns should work well with other RPM based distributions, and probably apply to other packaging systems as well.

Series index

(will turn into links as series is written)

  1. Introduction to RPM
  2. Building Basic RPMs
  3. Working RPM management software
  4. Patterns: Building configuration packages
  5. Patterns: Replacing vendor RPMs
  6. Patterns: Building RPMS for “trouble” software
  7. Conclusions?