Skip to content

Getting stfl working on fedora 11 x64

Newbeuter is my preferred way to browse RSS feeds. Its a great CLI app that is ‘Mutt Of Feed Readers’, and being a user of mutt I love this app for being so similar. I wanted to use the newest version of newbeuter, because recently I have been on a kick to get all of my apps: screen, vim, mutt, and now newsbuter; setup with 256 colours.

I read on the development blog that the newest version in the git repo was able to use 256 colours, so I immediately cloned the repo and started the install process. I was stopped immediately at the config.sh. It wanted the stfl libraries and wasn’t able to find them, and neither could yum.

Looking into the config.sh I saw that it was querying pkg-config for the flags to use on compilation to see if the library was present. It wasn’t, and so I also read further on the notes for the git repo version, and saw the bit about using the latest svn copy of stfl. So I checked that out, made it and installed it onto my machine. But this didn’t get me any further into making newsbueter, because pkg-config was still unable to find libstfl.

So I decided to try and figure out what the issue was with stfl and pkg-config. First thing I saw was that the stfl.pc file that pkg-config uses to make its responses was in the wrong place for my system. It was in /usr/local/lib/pkgconfig, instead of with the other libraries on my system in /usr/lib64/pkgconfig. This resolved the issue of pkg-config not knowing where the pc file for stfl was, and pkg-config was now returning actual data about libstfl. A recompilation of newsbeuter did show that there was still a problem. I now received: error while loading shared libraries: libstfl.so.0: cannot open shared object file: No such file or directory.

I did another updatedb, and ran a locate for all stfl files, and found that the make install for stfl was putting the libs in a strange place as well. I then updated both the Makefile.cfg and the stfl.so.in to reflect the locations of other libs:

Makefile.cfg changed lines

export libdir ?= lib64
export prefix ?= usr
export DESTDIR ?= /

stfl.pc.in changes lines

prefix=/usr
exec_prefix=/usr
libdir=/usr/lib64
includedir=/usr/include

This got me pretty far I think, because now the system looked like the other libraries. The stfl.pc lines were taken almost verbatim from the sqlite3.pc file in fact. I was still getting the shared library issue though, and after not thinking about the problem for a bit I realized that the stfl makefile had made libstfl.so.0.21 and the error message was not able to find libstfl.so.0, so I too the logical leap and made a symlink named libstfl.so.0 that pointed to libstfl.so.0.21 in the /usr/lib64/ directory. After making sure that this worked, I made a change to the makefile for stfl, and added another line to make a second symlink, that ended in 0:

Makefile

ln -fs libstfl.so.$(VERSION) $(DESTDIR)$(prefix)/$(libdir)/libstfl.so.0

I do admit that there may be a better way to do this, and some of it may be done with command line config or make flags. I don’t know them, and was able to successfully build with these changes.

Posted 30 July 2009 † Morgan § Linux ‡ Comments (0) °

Decouple with kwargs

So I’ve been attempting to make a suite of cli scripts for work. I recently discovered the multiprocessing module for python, and really liked its simplicity, and started using it, with great success. Everything was faster.

This then spurred me to take the scripts that were for the most part, copy common-ish bits and then modify to suite, and turn them into a library of sorts. The neat part then arose when I wanted to import a argument parser, as well as pass off to a proc creation component.

In doing this I had in mind that the ‘script’ would need to only define a function to make a list of commands to run on a given server, and a __main__ section that would pass in a list of servers, the function to make a command list and some other info. This way the script itself would only be two definition sections, and only the parts that were going to be unique for the most part.

The problem that came up in doing this is when I wanted the function that makes the command list to have more arguments that normal. How would I pass them in, and how would I define them so that I don’t have to edit my libraries to accommodate this argument passing. It was kwargs that saved me there, that and some optparse tweaking.

Here is a basic example:

script.py

def get_commands(host, **kwargs):
    command_list = []

    user = kwargs['user']
    key_file = kwargs['key_file']

    command_list.append("echo %s" % user)

    return command_list    

if __name__ == '__main__':
    import automation

    hosts = []

    (hosts, options) = automation.process_args(sys.argv)

    automation.thread_hosts(
            hosts,
            get_commands,
            options,
            user="test"
            )

automation.py (library w/ functions)

def thread_hosts(hosts, get_commands, options={}, **kwargs):
    import multiprocessing
 
    kwargs.update(options)

    jobs = []

    for host in hosts:
        p = multiprocessing.Process(
                target=run_commands,
                args=(
                    host,
                    get_commands(host, **kwargs),
                    ), )

        jobs.append(p)
        p.start()

So this example is a script that defines the function to return a command list, and provides an options var, and list of hosts. The thread hosts then loops over the hosts each time passing the host and the get_commands function to another library function that connects to said host, and loops over the returned command list.

A part that might be confusing is that the parse_args function returns optparse’s options variable but the options.__dict__ representation specifically. This then allows me to be able to update kwargs with any options that I allow to be set at the command line. The example in the script being the key_file variable.

The neat part of all this is being able to take the kwargs for one function and pass it right along to the next. This is key, because it allows for the library function in this case to be able to be entirely decoupled from the script itself.

With this implementation I am able to write a script that defines extra args to use, and only the script need know what they are. In the examples the library will just dumbly pass them along in the kwargs dict, I never have to tell it that I want to pass a user variable to it, and it makes the script a nice self contained unit.

Posted 22 July 2009 † Morgan § Linux § Programming ‡ Comments (3) ° Tagged: , , ,

Trac makes my life easy

The project management app Trac is something that was new to me a while back. I’d just installed t for a side project, and used the yum install without any issues. It took care of all the grunt work, and got me to the point where I could now create and use a trac project.

Trac is set up like what I see web frameworks go with. A main program that will install the framework in a project directory. In this case trac-admin , which is killer when you want to make multiple projects, and offers a cli interface to the project’s framework configuration, etc.

This setup becomes awesome I found when you want to upgrade. Yum installed what it had packaged, the .10 version, but I had decided that I wanted to toy with bitten their automated build tool, which required .11 and up. So an upgrade was needed, yum couldn’t be used, but I found that trac-admin has an upgrade command.

So I was poised to make the fun and scary transition into mixing a package managed install with a source install, not something that always goes well. I’ve found that sometimes packagers change to install location from where the src install goes (looking at you nagios), and make some conflicts or at least confusion.

The upgrade process for the server then my app was as simple as:

wget http://ftp.edgewall.com/pub/trac/Trac-0.11.5.tar.gz
tar zxvf http://ftp.edgewall.com/pub/trac/Trac-0.11.5.tar.gz
cd Trac-0.11.5
python setup.py install
trac-admin /path/to/project upgrade
trac-admin /path/to/project wiki upgrade
/etc/init.d/httpd restart

This blew me away. I’ve have never had a complicated app (relativily of course) upgrade so simply, and without any issues. The main install of trac from empty folder to working project manager was simple too, so perhaps I should have expected this, but really I think it is a testimony to how well the developers of Trac have though of the whole process of using their framework.

Links:
http://trac.edgewall.org/
http://bitten.edgewall.org/

Posted 22 July 2009 † Morgan § Linux § Webservers ‡ Comments (0) ° Tagged: , , ,

Authenticating svn and trac with wordpress

Problem caused by wordpress upgrade

My club uses WordPress and I have our forums and subversion authenticate via the wordpress install’s user table. This became very useful, and something that I tried to make sure I could apply on any new app I would install for the site.

When trying to get the same thing setup for an install of Trac I had just made I ran into a bit of trouble. With the old versions of wordpress this was pretty simple to do. Just a few lines in an apache conf file and we were golden.

With the most recent revisions though their implementation of password storage changed, causing the old setup to break for svn, and causing me a nice headache when trying to duplicate my old fix for svn onto trac. They went from a simple md5 hash to using a much more secure phpass. (why it isn’t phppass I don’t know)

The main problem with this is that this isn’t an authentication encryption that apache’s mysql handler could use. I tried to find a work around to get back to md5, but I couldn’t find any. It was probably for the best anyhow, as I’d rather the site be more secure, than have more tools. No point in propagating something that could be exploited. Searching around some more I found the awesome work of Nikolay. given out on Barry’s blog, and explaining the install process.  Nikolay made an apache module to compile that added in the ability to use phpass. This compiled great and worked with the fedora install the server is on,  so the old fix for subversion was working again, with a single line changed.

Subversion

RedirectMatch ^(/repos)$ $1/
<Location /repos/>
     Options all
     DAV svn
     SVNParentPath /repos/gcc/
     SVNListParentPath on

     AuthName "MySQL authentication for SVN"
     AuthType Basic
     Require valid-user

     AuthMYSQLEnable on
     AuthBasicAuthoritative off
     AuthMySQLAuthoritative on

     AuthMySQLHost localhost
     AuthMySQLUser user
     AuthMySQLPassword password
     AuthMySQLDB wordpress_db
     AuthMySQLUserTable wp_users
     AuthMySQLNameField user_login
     AuthMySQLPasswordField user_pass
     AuthMySQLPwEncryption phpass
</Location>
CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

That is the config that makes sure that only people that have accounts on the wordpress blog can have access to the repos. I plan on soon adding in a SVN auth file to make the commit users  more constrained, but at the moment, it isn’t a priority.

The last line makes nice entries of SVN access in its own log file, which is very handy for debugging problems.

Trac

For trac I took the simple apache auth they provided on their website, and applied the same idea from svn to it:

<Location "/projects/project-name/login">
     AuthType Basic
     Require valid-user

     AuthName "Trac Auth"
     AuthMYSQLEnable on
     AuthMySQLAuthoritative on
     AuthMySQLHost localhost
     AuthMySQLUser wordpress
     AuthMySQLPassword password
     AuthMySQLDB wordpress_db
     AuthMySQLUserTable wp_users
     AuthMySQLNameField user_login
     AuthMySQLPasswordField user_pass
     AuthMySQLPwEncryption phpass
</Location>

I plan to use this type of database integration more, specifically with a wiki installation. Although I don’t know of any wiki that could use this type of authentication as I am only familiar with mediawiki, and really only as a user.

Links

  • http://barry.wordpress.com/2008/05/19/mod_auth_mysql-and-phpass/
  • http://nikolay.bg/
Posted 07 May 2009 † Morgan § Webservers ‡ Comments (0) ° Tagged: , , , , ,

Fedora 10 on the eeePC 900

Recently got the woot.com deal on the eeepc 900, sans webcam. I tried to use the xanadros os that was bundled with it for a few days, but the lack of normal gnu tools and normal packages made me want to switch. So I decided to to look into putting my os of choice fedora onto it.

Installing:

Found that is was a dooable project, exciting and surprising.  Since the netbook doesn’t have a dvd-rom drive, and I lent out my external, I decided to go the usb live stick route.

Directions for this were easy to find, I did have to use the command line version of them, and found through trial and error that only my name brand Kingston drive would get the netbook to boot.

To get the usb drive to boot, hit escape while booting or hit f2 and set the main drive to be the usb drive.  After it boots and finishes and installs, everything pretty much works.

When I brought up the machine I started to follow Gavin’s directions and run a general update. This failed, only because it filled the small hard drive up. I then had to find the /var/cache/yum/updates/packages/ directory and clear it by hand, because yum didn’t have the room to run and remove the rpms itself.

Removing things to make space:

So now I had to sleuth around to find out what was taking up all the space on the drive. Using

du -Hs -si /*

found that the largest folder in the root dir was usr. And walking down that path structure I found that the three largest folders, that I could do something about, were:

  • /usr/share/locale @332MB
  • /usr/share/doc @106MB
  • /usr/share/fonts @223MB

So I really went through and removed a whole lot of this stuff, which depending on ones needs, may not be advisable.

I removed all fonts that could using yum instead of just deleting them, as that just really seemed like a bad idea. It took a bit of grep’ing, so this line is a little long, but it basically strips out all the extra fonts, and keeps liberation as well as core font tools and libs. I personally use Anonymous, for all my coding, and terminal work, and I put that in .fonts dir, so really even this is a bit liberal for me.

rpm -qa | grep font \
| grep -ve'xorg' -ve'core' -ve'liberation' -ve'fontconfig' -ve'lib' -ve'bitmapfonts' -ve'ghostscript' \
| xargs yum -y erase

Note also that this removes these apps:

  • gimp
  • abiword
  • evince
  • ghostscript (even though I tried not to)

This frees up most of the 200MB that the fonts dir takes up, so we’re off to a good start.

The locale directory I was unfamiliar with, but some searching found that this is where data for other language support is stored. I only speak one, and found that removing these doesn’t cause irreparable harm. So delete them I did.  Since these are installed with every program, and the language packs don’t have their own RPMs yum wasn’t useful in this case.

ls /usr/share/locale/* -d \
| grep -ve'locale.alias' -ve'default' -ve'en' \
| xargs rm -rf

This frees up about another 300MB. But it is also something that will crop up again, since any new program you install will add it own locale information. For apt-get there is a nice plug-in that will automatically strip out this information for you on install, but in my shallow searching for this for yum there didn’t seem to be a substitute. Might be a good weekend project.

The share doc directory I just cd’ed into and wiped out its contents. This is a netbook, and I can ssh into my home machine to read those if I need to, so I freed up another 100 or so MB with this wipe.

This brings us a lot closer to a good  1 GB of free space. I went further and removed a few apps I won’t be using:

  • evolution
  • rhythmbox
  • cheese

Now to try and get the services under control.

Found this great listing of what each service does, so if you’re unsure if you want to stop as many as I chose to, look it up and make sure. I also chose to use the command line service conf tool:

/sbin/chkconfig  --list

  So its a bit time consuming but I went through the chkconfig list and grepped out the 5:on states and checked them against my list below and set them to off one at a time for every run time level.  There might be a quicker way, but I couldn’t think of one that didn’t involve making a script, so I let it be, and stuck with manual.

  • cron, atd, anacron
  • auditd (also disabled SELinux)
  • avahi-daemon
  • bluetooth
  • btseed, bttrack
  • capi
  • cups[*]
  • firstboot
  • ip6tables
  • irda
  • irqbalance
  • isdn
  • kerneloops
  • lm_sensors
  • mdmonitor
  • multipathd
  • netconsole
  • netfs
  • nfs
  • nfslock
  • nmbd
  • nscd
  • pcscd
  • portreserve
  • restorecond
  • rpcbind
  • rpcgssd*
  • sendmail
  • smb
  • ypbind

Now not all of those services were up, but I just made the list of what I would remove, and didn’t take note of what was not running in that list.

Then I installed my must haves: xfce, vim, htop, all the dvcs’ , tilda, and tomboy. I rsync’ed over my dot dirs that I wanted, and I was good.

Links:
http://fedoraproject.org/wiki/EeePc#Eee_PC_90x.2F1000_Series
http://idolinux.blogspot.com/2009/02/fedora-10-on-eee-pc-1000.html
http://fedoraproject.org/wiki/FedoraLiveCD/USBHowTo
http://www.mjmwired.net/resources/mjm-services-f10.html

Posted 04 May 2009 † Morgan § Linux ‡ Comments (2) ° Tagged: , ,