Developers' Blogs: VMiklos

..continuing last year's article. so another year passed by and it's time to look back and see what we did during 2007. probably i miss a lot of stuff but here is my list:

1) ability to go back in the installer to a previous point if you missed something. do you remember the days when one had to reboot if he/she wanted to do so?

2) compiz improvements. this is now settled down in current and it's pretty sane. we cleaned up the old compiz and beryl, we have a single compiz-fusion, it has a nice step by step documentation and it works fine both for kde and gnome.

3) asciidoc. i think we highly improved our documentation since we switched from latex to asciidoc. a user manual of 98 pages in a nice pdf format is cute, isn't it?

4) newsletters. Alex started to issue newsletters and recently phayz helped out us, so it's alive again. i think it's something great.

5) yugo. 'factory' was our previous i686 build server, it was a very old machine with a cpu of 300mhz and so on. it was time to replace it and now yugo does the job.

6) fwlive. this was an old project but only test versions were available, based on old frugalware versions. now there is a live version of every released version of frugalware, thanks to janny, boobaa and ironiq. great!

7) gnetconfig. the first graphical config tool from priyank. i'm really bad at any graphical programming, so i'm glad to see finally we started to work on guis.

8) gfpm. something users always wanted and now it's here. a true graphical package manager, which is not just a wrapper but properly uses libpacman. awesome.

9) fun. this is our update manager which can sit on the system tray (or whatever i should call kicker not to be kde specific ) and notifies you if there is something to update. i'm sure this is more comfortable compared to watching the -security mailing list for updates or doing a -Syu daily

10) syncpkgd2. if you remember, the old method was that there were only clients and they tried to figure out what to build, they built and uploaded the packages. this was very suboptimal: it allowed only one buildserver / arch and it was slow. okay, being slow is the smaller problem, but every buildserver was a single point of failure. nowadays we have two i686 buildservers (thanks to boobaa) and it's theoretically it's possible to have two x86_64 buildservers, too. so even if one i686 buildserver is down, i can be at the beach, sipping a mojito

okay, i learned something today

you have a client behind a firewall and you have a server somewhere. and you want to ssh from the server to the client. that's exactly why ssh has a magic -R option!

if you do an

$ ssh -R 19022:localhost:22 server

that will mean that you can ssh to client:22 on the server by sshing to localhost:19022 on the server. yes, and it works even if the client is behind a firewall, yay!

update:

you probably want to add this to your ~/.ssh/config on the server as well:

Host client
        NoHostAuthenticationForLocalhost yes
        HostName localhost
        Port 19022

so that you can easily just type

$ ssh client

on the server

git-svnimport will be removed in git-1.5.4 and i don't think it's tirival to use git-svn (which is great for bi-directional work) to properly convert an svn repo (with multiple branches) to a git one, so here is a recipe.

first, just use git-svn to create a git-svn repo. it's a git repo but it's a bit weird, it contains metadata which is necessary for bi-directional operations and you won't see the branches by default when you clone it. ah and it's not a bare repo.

in this example i'll convert ooo-build's svn repo (it has several branches and tags, so it's a good example).

mkdir -p ~/git/ooo-build
cd ~/git/ooo-build
git svn init -s svn+ssh://svn.gnome.org/svn/ooo-build
git svn fetch

this will take a while. once it's completed, you need to convert this to a bare repo:

mkdir ../ooo-build.git
cd ../ooo-build.git
git --bare init
git config remote.origin.url ~/git/ooo-build
git config remote.origin.fetch +refs/remotes/tags/*:refs/tags/*
git config --add remote.origin.fetch +refs/remotes/trunk:refs/heads/master
git config --add remote.origin.fetch +refs/remotes/*:refs/heads/*
git fetch


and you're done!

of course this is an incremental operation, so all you need is to run git svn fetch and git fetch from cron to keep the mirror up to date.

update:

it turns out you can even do this by having only one git repo. it has benefints: you don't have to duplicate the object database. and it has one problem: if you later switch fully to git, then you want to remove the git-svn metadata - doing so is the easiest if you just clone the repo.

so - in case you want this, you have to:

mkdir -p ~/git/ooo-build.git
cd ~/git/ooo-build.git
git --bare init
git --bare svn init -s svn+ssh://svn.gnome.org/svn/ooo-build
git --bare svn fetch

and then you can fetch from yourself:


git config remote.origin.url .
git config remote.origin.fetch +refs/remotes/tags/*:refs/tags/*
git config --add remote.origin.fetch +refs/remotes/*:refs/heads/*
git fetch


note: git config --add remote.origin.fetch +refs/remotes/trunk:refs/heads/master it unnecessary as git svn fetch will update this for you (it is different from git fetch)

thanks for devurandom from #git for pointing out that this is possible using git --bare init before git svn init and using git --bare svn fetch

it was a while ago when we registered at ohloh.net, but i recently re-visited that page, and i found some nice PR texts there

Over the past twelve months, 18 developers contributed new code to Frugalware Linux.

This is a relatively large team, putting this project among the top 10% of all project teams on Ohloh.

For this measurement, Ohloh considered only recent changes to the code. Over the entire history of the project, 36 developers have contributed.

(source)

well, i always considered our team as a nice and good but small team, it turns out that if you count projects, we're not that small!

it also creates nice charts about what programming languages do we use, etc, etc. so it's a different approach, compared to cia, but it's interesting.

recently i needed to convert a cvs repo to git, in fact i'm providing a git mirror, so i need incremental updates.

here is the command i needed:

$ git cvsimport -d :pserver:anonymous@tcpflow.cvs.sourceforge.net:/cvsroot/tcpflow -C tcpflow tcpflow

yes, this will do the initial conversion and the incremental one, too.

tcpflow is a nice small console application to debug your network data. the usage is simple. just something like

# tcpflow -i eth0

is enough. you're recommended to start it in an empty directory

don't be surprised, the incoming and the outgoing connection to a server's given port will be logged to 2 different files

git filter-branch is a new command in git-1.5.3 and today i met a situation where it was really useful. here is the scenario:

i have a 'vmexam' (like soxexam, "vmiklos' examples") repo, random small code chunks, every commit touches only a given subdir. after some time some projects need an own repo and i do versioned releases. this was the case with pyrssi, too

so i wanted to have a new repo with only the pyrssi-related commits and without anything else. this is exactly the case git filter-branch's subdirectory filter is for:

git filter-branch --subdirectory-filter python/cgi/pyrssi HEAD

then in the old repo i just rm -rf'd the code and i had it in the root dir of the new repo. yay! :)

so, here is the situation:

you have a machine behind a firewall and you have an ssh session to a server. and you want to send notifications to you machine. here is what i did, maybe useful for others, too

• you'll need the program called osd_cat, provided by osd
• you'll need to use ssh with the -X option
• if you run programs inside screen, you still won't know the the right DISPLAY number, so do a trick: add

  [ -n "$DISPLAY" ] && echo $DISPLAY > ~/.display

  to you ~/.bash_login

• now you're ready to use osd_cat:

  $ echo -e 'foo'|DISPLAY=`cat ~/.display` osd_cat --color=blue --age=4 --font=-*-helvetica-medium-r-*-*-*-320-*-*-*-*-*-* --offset=100 --align=right --indent=100

as an example, irssi has osd.pl to use osd_cat for highlight notifications

Priyank recently removed Beryl from -current and added Compiz Fusion. so i did a base install then i followed the instructions in xgl's README.Frugalware. a few impressions:

pros:

• i have an i810 card, not nvidia, or anything extra, it's a cheap one. the only effect that was slow for me is when i marked the cube 100% transparent. but all the other effect were really fast.
• compiz initially worked fine only with gnome. i'm a kde fan so i tried compiz with kde and it worked fine. i think this is something new
• thanks to aiglx, i did not have to - in fact - start xgl, no editing in kdm.conf or anything, just xorg.conf

cons:

• i have 4 virtual desktops like:
  

  1 | 2
  ----
  3 | 4

  compiz translates this to: a simple paper with two side and on each size there is another side on top of the other one. so it says it's not a cube. how crappy

• i can't select to hide windows from other virtual desktops from the taskbar. a basic feature @ kde

conclusion? it's really nice, well done - but for daily work of course i won't enable it

here is the problem:

you hava gpg key since 5 years. you had various mail addresses over the time and every mail address is attached to a key. so it'll look ugly, like this.

the problem is that once you pushed a uid (read: email address, attached to a key) to the keyserver, you can no longer remove it. but here is the solution: you can revoke it!

how to do so?

it's easy:

$ gpg --edit-key john@foo.org

a list of your ids with numbers are displayed. select a uid:

Command> uid 2

then you can revoke it:

Command> revuid 3
Really revoke this user ID? (y/N) y
Please select the reason for the revocation:
0 = No reason specified
4 = User ID is no longer valid
Q = Cancel
(Probably you want to select 4 here)
Your decision? 4
Enter an optional description; end it with an empty line:
>
Reason for revocation: User ID is no longer valid
(No description given)
Is this okay? (y/N) y

you can notice that the uid is now listed as "revoked" instead of "ultimate"

now you can exit using:

Command> save

and send it to the key server:

$ gpg --send-key 03915096

of course the uids will be still there, but that bloated list should be now short and tiny

maybe you know a situation that is regular for me: when testing a new wireless network - if dhcp is misconfigured, dhcpcd would just have to timeout. though - thanks to zeroconf support - it uses to pick up some ip addresses instead, making debugging hard. so here is a trick to disable zeroconf support (so that dhcpcd will just timeout):

append

dhcp_opts = -L

to your /etc/sysconfig/network/default (or other) profile for the section of the wireless interface

dht support means support for trackerless torrents

currently rtorrent does not support it, but after a bit googling, i've found this ticket. it mentions that there is a patch here. so i've split up the patch for libtorrent and rtorrent and uploaded these packages to my bmf repo.

you need to add two extra lines to your ~/.rtorrent.rc:

dht = auto
dht_port = 6986

of course the port can be anything else.

during testing, i had one "server" - where i had the test file already - and one "client" - where i had only the torrent.

i opened one port for torrent communication and one port for the dht server on the server, of course no ports were open on the client. and yes, it did work fine!

(screenshot)

let's hope the patch will be included in the vanilla rtorrent soon!

it's a nice simple script to generate a message body for you when you have no write access to a repo and you want to request the m8r to pull from you. here is how to use it:

$ git request-pull origin/master \ 
vmiklos@vmiklos.dev.frugalware.org:git/pacman-g2/
The following changes since commit 52a29e40a52bb09e0db7f167712a9be3ad8d0d3c:
  Priyank (1):
        Merge branch 'master' of \
git.frugalware.org:/home/ftp/pub/other/pacman-g2/pacman-g2

are available in the git repository at:

  vmiklos@vmiklos.dev.frugalware.org:git/pacman-g2/ master

VMiklos (4):
      new NOARCH trans flag for _pacman_pkg_load()
      _pacman_pkg_load(): indentation fix
      parseargs(): new --noarch option
      tests/upgrade071.py: new test for --noarch

 lib/libpacman/package.c     |   36 +++++++++++++++++++-----------------
 lib/libpacman/pacman.h      |    1 +
 pactest/tests/upgrade071.py |   10 ++++++++++
 src/pacman-g2/pacman-g2.c   |    3 +++
 4 files changed, 33 insertions(+), 17 deletions(-)
 create mode 100644 pactest/tests/upgrade071.py

of course in real life, you want to use http:// or git:// since probably the m8r won't be be able to login to your own machine via ssh

after a long time, openoffice now builds again out of the box without any patches as all our fixes are in ooo-build

also, i recently started to re-use my "bmf" repo for packaging pre/rc versions, so it was logical to get rid of the "ooodev" repo and merge it to bmf. so openoffice.org-2.3oog680_m5-1 (alias 2.3.0rc3) is available for my bmf repo for both i686 and x86_64. compared to 2.2.x, the two notable improvements (imho) are the working mozilla plugin and the ooxml support.

thanks boobaa for building it for i686!

if you use rsync to copy your git repo (i mean a non-bare repo here: a repo with a working dir + a .git one), git diff will show that you touched your files. using git checkout -f can be a workaround to hide those changes, but then when you rsync again, it'll see that git thouched those files, so it'll re-download all the tracked files. infinite loop. so here is a trick from gitte: using

git update-index --refresh

the index itself will be updated, not the tracked files. this was rsync will only re-download the index which is far less time, compared to the original one

i've uploaded an updated mplayer package to my repo which includes dvdnav support to mplayer. at the moment it seems you need to

# pacman -R codecs

otherwise mplayer will crash

many thanks to Ötvös Attila, the dvdnav patch author for his help (two additional build fixes was needed for our "enable everything what is possible" mplayer configuration)

hopefully we'll figure out soon how to resolve the conflict with the binary codecs, too

(a screenshot)

i think using git-svn is really easy, once you figured out how to use it. so here is how i use it. no, i'm not a git-svn expert, just a user. and the way i use it - i think it's easy. so here is our situation:

1) foo project uses svn for managing their code
2) you wish they would use git

how to do both:

1) get the full svn repo:

$ git svn clone --prefix=origin/ url

yes, in my situation this project does not use the trunk/tags/branches structure, see the git-svn manpage if the project you clone does (s/clone/clone -s/ should do the trick)

the --prefix parameter is needed so that you'll see the svn branches as if they were normal git remote branches

2) your local repo is a git repo, so you can do whatever you want, commit, revert, bisect, etc

3) instead of git pull, you need

$ git svn rebase

this does the same as git fetch + git rebase (since you can't merge because of svn)

4) instead of git push, you need

$ git svn dcommit

and we're ready. 3 and not more command, right?

this is always a subject of flamewars and Frugalware did not have any support for coloured manpages so far. i've now added (in man-1.6e-2) support for it, but it's still disabled. see the documentation on how to enable it if you like. you can reach a screenshot here if you use the default colors.

when we switched to git, our darcs stat page was no longer updated. now i've created gitstats, which serves the same purpose. it's written in python and it was a few more features compared to darcsstats, ie. alias support, so that the stats for a dev are not lost when he/she changes his/her email addr.

the new stat page is here. enjoy!

well, it truns out what i'm mentioned at bitlbee.org

the story is about my skype plugin for bitlbee. google doesn't even find it (mostly because nobody links my page), so i asked the devs to link the page. now they posted a news entry about it

/me goes to sleep..

i've recently pushed a new asciidoc to my repo. after intalling it, you have to be able to display this example page properly. (here is the source)

so now go and tell your latex friends that you can still include your exotic math formulas in asciidoc while you can avoid its usualy difficult syntax!

as of openoffice.org-2.3oog680_m1 there is a firefox plugin (for OOo documents) included in our openoffice pkg. at the moment it's not enabled by default, you need an

ln -s /usr/lib/openoffice.org/program/libnpsoplugin.so ~/.mozilla/plugins

to enable it

long time no blog

in darcs, tags are empty patches which depend on every patch since the previous tag + the previous tag. so tag is shown in the changelog itself

in git, tags are just pointers to existing commit objects. for example 1c77ec3 already exists, then you say let's call this 1.0

you may want to create darcs-like tags in git. of course this won't be the git way, but it's nice to see that the git core allows you to do so

so here is what you need:


echo "tagged 0.9.2" |git update-ref HEAD `git commit-tree HEAD^{tree} -p HEAD`


thanks hliusv561!

ok, there is no such option, but there is an equivalent trick

for example if you want to do exclude the _darcs dir, then you need:

find . -type d -name _darcs -prune -o <filter> -print

where <filter> can be "-name *.c" or so

thanks crazy for the tip

i've recently got a mail about linuxquestions.org now has a separate collection of Frugalware screenshots. so here it is, maybe one find it interesting