Bash conditional

I have been always confused by the Bash conditional tests. This link is a great
tutorial.
http://www.linuxtutorialblog.com/post/tutorial-conditions-in-bash-scripting-if-statements

Running cron in cygwin

I setup my cron job in cygwin today. The correct way to start the cron
daemon is using the /usr/bin/cron-config tool that comes with the cron
package. You can choose to run cron daemon with you own user account
or as a SYSTEM account process. The difference is explained in
/usr/share/doc/Cygwin/cron-VERSION.README. I chose to run cron
as SYSTEM process. This gave me a bit trouble. Because the job spec
I edited is owned by my own user account. The cron daemon process
cannot load a crobtab file with different owner than its process owner.
(Maybe not correct, but once I change my crontab file to be owned by
SYSTEM, it worked). I got this fixed by using the cronevents.exe tool,
which is really really helpful. Also, you might want to cron_diagnose.sh
script to check if various configuration is correct, such as the /var dir's
permission.

Svn repository mirror using svnsync

Because my company's main svn repo is in a different city, we constantly
suffer from the slow network during development. The idea of setting up
a read-only mirror has come to my mind for a long time. But I haven't had
the time to implement it. Today, I spent half a day to set it up finally.

Here is the note to record what I read and did to make it work:

1. Read this post It tells almost everything you need to know to set it up, of course, you need to have some basic svn knowledge.
2. For me the sync was not working due to failing the acquire a write-lock. The error message is: "svnsync: Can't open file '/var/svn/pim_12_mirror/db/write-lock':Permission denied". I fix it by chowning everything under the repo to be owned by svn user.
3. Because the mirror is read-only. Using 'svn switch' sub-command will greatly help you to get the best of the mirror while being able to commit the original repo.

Vim with mlcscope

It's been a while since my last post...

You know, my IDE is bash + Vim + exuberant ctags + cscope + GNU
global(optional), normally. This is almost perfect when I am doing C/C++
development. But now, I spend quite a lot of time doing Python programming.
ctags for python is not enough. GNU global is not supporting Python at all.
So I am looking into the multi-lingual verion of cscope, that is, mlcscope.

But getting Vim to work with mlcscope is a bit problematic. I searched Google
for the solution. The top post is this one, which is dated and no good solution
provided. So what I am going to do here is to put my solution into the
WWW database. To let Vim use the cscope.out file generated by mlcscope,
you just need to "set cscopeprg=mlcscope" in your vim start setting file.
Obviously, the cscope.out file generated by mlcscope cannot be read correctly
by the original cscope.

Command to check cygwin installation information

Command to check cygwin info:

cygcheck -s -v -r -h

It dumps:

1. Windows system version
2. Windows system path
3. User ID
4. System directories and variables that affect cygwin
5. Environment variables when you run in cygwin
6. Cynus related registry entries
7. Mount drives
8. Cygwin entries for the mount drives.
9. List important binaries locations
10. List Cygwin DLLs' information
11. Installed cygwin services
12. Cygwin packages information

Binmode, textmode, cygwin, msys and grep

In the UNIX world, I guess, not many people care about the file is opened
in text mode or binary mode. In fact, most of us probably are just using
binary mode. But when it comes to the Windows world, the difference is
becoming a pain in the arse. Because in Windows, when you open a file in
text mode, every time you write a line of text, Windows makes it end with
'CRLN' instead of 'LN'. I don't intend to make a detailed explanation here
and reinvent the wheel. The best explanation is here:
http://www.cygwin.com/cygwin-ug-net/using-textbinary.html
But that link just talk about cygwin, how about MSYS. Is MSYS doing the same?
Yes, it is. By default, MSYS is in binmode. So whatever is passed in will
be pass through (the shell here I mean, including the pipes, redirected file
descriptors) untouched. So what I was trying to find out and puttting down here
is: If you run a Python program (using Windows python), the output pass through
a cygwin/msys shell (pipe), what is the end-of-line marker? My non-authorative
answer is: whatever the python program outputing and when you run the Windows
version of python, every text line is output with 'CRLN' end-of-line'.
So this is 'fine', as long as it is consistent on Windows. The thing is driving
you mad, when some programs is doing the correct thing, that is, stripping the
'\r' off the input/output lines. Which programs fall into this category, I now
find one for sure, that is grep, one of the most useful POSIX tool. The MSYS grep
will silently strip off the '\r' (CR) on each line, which the cygwin one will not.
Imagine what this will cost you! So to be safe, I decide to always use the
-U (--binary) switch.

Below is a test program to verify this. I wrote a python script:

import os

print 'hello world'

f = os.fdopen(1, 'wb', 0)
f.write('hello world\n')
f.write('hello world\r\n')

import sys
sys.stdout.close()
sys.stdout = f

print 'hello world'

Run it with command:

python test.py | xxd

See what is the output!

Run it under MSYS with this command:

python test.py | grep -v 'asdf' | xxd

You will see it is end with 'LN'.
Now run it with this:

python test.py | grep -U -v 'asdf' | xxd

You will see it is end with 'CRLN'.

Try the two commands on cygwin, both have the same output.

Open cygwin/msys shell from Explorer window context menu

If you are a cygwin/msys user on Windows like me, you may find the
need to open a shell window from the Windows explorer. This can be
done via a context menu entry for a folder. Right-click on a folder,
and open a shell window with that folder as the current working
directory. That will be really nice, right. Windows itself also
has a toy called cmdhere, which does the
similar thing but brings up the cmd.exe window.
So how to do the cygwin/msys shell window? I search around a bit.
There are many post on the Web. But I think I settle down on this
simple one. First you need to write a bat file that take one
argument, which is the working directory. Second, you need to
modify the registry entry to add context menu for this bat file
for the folder type context menu. The bat files for my cygwin
shell is like this:

@echo off
start /min C:\cygwin\bin\bash --login -i -c 'cd "`cygpath %1`"; rxvt +sb +vb '

The msys version bat is like this:

@echo off
start /min C:\cygwin\bin\bash --login -i -c 'cd "`cygpath %1`"; rxvt +sb +vb '

Then add a new key under the "My ComputerHKEY_CLASSES_ROOT\Folder\shell"
registry entry. The new key's name can be something lik Cygwin_here/msys_here.
Then add a new key called "command" under this new entry, with the value
to be your bat file with the "%1" argument. My exported registry entry is
like these:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Folder\shell\Cygwin_Here]
@="Cygwin Here"

[HKEY_CLASSES_ROOT\Folder\shell\Cygwin_Here\command]
@="C:\\cygwin\\rxvt_here \"%1\""

[HKEY_CLASSES_ROOT\Folder\shell\Msys_Here]

[HKEY_CLASSES_ROOT\Folder\shell\Msys_Here\command]
@="c:\\msys\\1.0\\msys_here \"%1\""

How to unload a python extension dll

I haven't found the answer. But I would like to put here
as a place holder. Also hoping if an expert can comment
on it. :)

Basically, the problem is: I have a python extension
written in C/C++. As a python module, I import it
in my interactive shell (IPython indeed). I modified
the C/C++ code, rebuild the extension, and want to
test it out in my python shell. But even I deleted the
module from the interactive shell namespace, the python
process is still holding a handle to the DLL (pyd). I
have to kill the python process and restart the testing
from beginning again. It will be nice if I can ask python
VM to release the handle of the pyd module and re-import
the module again.

Can I do it?

How to find out system information on Window

Sometimes, we need to know the properties of the OS we are running on,
such information of interests include, page size, etc. On Linux this
is quite easy to get, because Linux has the proc file system to export
kernel information to user land as a file systems. On Windows I haven't
known a good and easy way. But Win32 API has a GetSystemInfo call,
which fulfills most of the need. Below is the sample code to print out
the page size of Windows.

#include <stdio.h>
#include <windows.h>
 
int main() {
 SYSTEM_INFO si;
 GetSystemInfo(&si);
 
 printf("The page size for this system is %u bytes.\n", si.dwPageSize);
 
 return 0;
}

GVim diff mode integration with svn on windows

How to use GVim diff mode as the diff viewer for svn
has been a research task for me for a long time.
Today, finally, I got a chance to devote some time
to set it up once and for all. On Windows, I use
two svn client. The GUI Tortoise svn client, and the
command line CollabNet client.
For the GUI client, it is quite easy to set up. Fire up the Tortoise svn
settings diaglog. Change the Diff Viewer setting to where your Windows GVim exe
is. Put the command line swith '-d' following the exe location. That is it.
Mine is like this:
C:\Program Files\Vim\vim72\gvim.exe -s "C:\Program Files\Vim\vimfiles\scripts\vimdiffSize.vim" -d %base %mine

Why I have another '-s' swith with argument? That's because
I want the diff window to be big and the two diff window split
equally. There may be other ways to do it. But mine way is giving
a vim command script to do it. The content of the command script
is like this:

:winsize 300 100^M
^W=

The '^M' and '^W' is one character, Control-M and Control-W. How to
enter it? In Vim type: Ctrl-V Ctrl-M and Ctrl-V Ctrl-W.

How about on the command line. svn has a config file. On Windows
it is in: c:\Documents and Settings\$UserName\Application Data\Subversion\config.
In this config file you can specify the option for diff-cmd,
which is equivalent to the --diff-cmd swith for 'svn diff' command.
The argument for this option need to be a Windows executable. So
you need a bat file to wrap you GVim exe. My diff-cmd option
setting is like this:

diff-cmd = "c:\msys\1.0\home\khong\bin\svndiff.bat"

The content of the svndiff.bat file is as following:

@ECHO OFF

REM Configure your favorite diff program here.
SET DIFF="C:\msys\1.0\opt\GVimPortable\gvim.exe"

REM Subversion provides the paths we need as the last two parameters.
REM These are parameters 6 and 7 (unless you use svn diff -x, in
REM which case, all bets are off).
SET LEFT=%6
SET RIGHT=%7

REM Call the diff command (change the following line to make sense for
REM your diff program).
%DIFF% -s c:\msys\1.0\home\khong\.vim\scripts\vimdiffSize.vim -d %LEFT% %RIGHT%

REM Return an errorcode of 0 if no differences were detected, 1 if some were.
REM Any other errorcode will be treated as fatal.

I am not the author. I got it from the (I think) svn official site.
The original name is svnwrap.bat.

Anyway, now I can use GVim diff mode to view diffs from svn.

Setting up command line mail client on Windows

Command line email client? There is nothing wrong with it if you hate the
sluggishness and tediousness of firing up a slow GUI email client.
So I set up mutt/fetchmail/procmail/ssmtp on my Windows box (on cygwin of course).
This is the post I follow to set them up. Works like a breeze!

http://www.gentoo.org/doc/en/guide-to-mutt.xml

Firefox plugins

You know, I am a hardcore Vim user. I even want my browsing experience to be
like Vim as much as possible. There are two Firefox plugins that I highly
recommend.

The first one is "It's all text!". It is a Firefox extension that allow you
to edit textareas in a webpage in external editor of choice. In my case
of course it is Vim. :) "It's all text!" homepage is here:
https://addons.mozilla.org/en-US/firefox/addon/4125


The second one is Vimperator. It is live changing for me.
It can be found here: http://vimperator.org/trac/wiki/vimperator

Happy Vimming!

How to make IPython shell Ctrl-h working on Windows

Although I am a vim user. I like the shell emacs binding.
I can't live without it. I have been using IPython a lot
on Windows. IPython's pyreadline is marvalous. But it has
one shortcoming (defect?). That is the 'Ctrl-h' key doesn't
work like 'backward-delete-char'. I have been searching for
a solution, for a long time. Only when I found this post,
has the problem been sovled:
http://makunouchi.jp/zope3/9900417093

:) It is in Japanese. But who minds. Apply the patch to your
pyreadline's keysyms.py file. Ctrl-h is back!!

Post the patch below if the Japanese link scares you.

--- pyreadline/keysyms/keysyms.py.orig
+++ pyreadline/keysyms/keysyms.py
@@ -119,6 +119,10 @@
        char = chr(VkKeyScan(ord(char)) & 0xff)
     elif control:
         char=chr(keycode)
+    if control and ord(char)==8 and keycode==72:
+        keycode=8
+        control=False
+        state &= 0xfffffff7
     try:
         keyname=code2sym_map[keycode]
     except KeyError:

Explanation of why Windows console-based app hangs on Rxvt

http://code.google.com/p/mintty/issues/detail?id=56
This post is clear and comprehensive. I am glad I found it.

But it doesn't solve the problem totally... I can now run
ipython shell in the msys Rxvt terminal. But the pyreadline
key bindings are screwed up totally. :( Still stay with Console2
for now.

Productivity Tool on Windows (For the GNU/Linux minded) -- Third

Today, I want to recommend a graphical diff tool on Windows.
I think most developers will always have their favorite diff
tool. But if you need to develop on 3 or more platforms, for
example, Mac OS X, Linux and Windows, having one favorite is
not that easy. If you are also using multiple revision
control systems on all these platforms. It will be even
harder. I am a Vim user. Whenever possible I use Vim, even
diff with Vim. Vim can run on all the above OSs. The only
problem I found is using Vim diff mode with different
revision control systems. I need to write script to let Vim
integrate with different revision control system. I am a
lazy person. Even though I like Vim, I don't like this
inconvenience.

So I heard about Tkdiff. Tkdiff is written in Tcl/Tk toolkit. You may
have a little despisement in your heart, right? Well, as the Chinese
saying goes: A man can not be judged by his appearance. Tkdiff is, in
my opinion, powerful, yet simple. The features I found useful are as
following:

1. Understand multiple revision control tools. Can diff different
versions of the file (work with revision control tools).
Currently tkdiff can work with cvs, subversion, perforce out of
the box. But with some patch, it can work with bazaar and mercurial.
2. Easy navigation. Can navigate to differnt diff region easily with
single key press, like 'N', 'P', 'C', etc.
3. Can integrate with external editor, such as gvim to edit the file.
4. Easy refreshing the diff. Just one key press 'R'
5. Run on multiple platform. As long as there is Tcl for that platform.

In a word. TkDiff is a marvelous tool, with humble appearance and name.

Installing pyQt

Today I installed PyQt on both Windows and Mac. The Windows version is quite easy.
It is just an executable installer. Run it and that is it. The Mac version takes me
a bit of effort. I follow the instructions on this link
(http://kdl.nobugware.com/post/2008/06/05/PyQt-4-on-Mac-Os-X/). You may need to
search for the newest packages since this post is a bit old now. But anyway,
it works. The only hickup is: I am also using fink packages on my Mac and there is
qt3 fink package installed. I need to tweak the path to eliminate the '/sw/bin'
and '/sw/usr/bin' (the fink's binary path) from PATH before I ran the configuration
step. So the configure.py script will not think it find the Qt installation in
the fink package locations. OK. Time to learn some PyQt and wxPython.

Productivity Tool on Windows (For the GNU/Linux minded) - Second

It is great to have a proper terminal window and proper shell on Windows. But
sometimes, this is not applicable to run some programs that relying on the
cmd.exe terminal. Python and iPython for Windows unfortunately falls into this
category. The native Windows Python build (not Python in cygwin) has to be run
in cmd.exe terminals (so is the iPython Windows native build, which runs with
the native Python). If we run Windows Python within a Rxvt (or xterm)
terminal, it just hangs without any input and output. Yuck! What is the
problem? I don't really know, either. (If someone knows, please tell. Better
if you have a good hack to solve it :). But anyway, I have to run Windows
Python and iPython, and I cannot use Rxvt for that. So it triggers my search
for a better cmd.exe terminal that won't hang Python. The result is
Console2! http://sourceforge.net/projects/console/. In Console2 you can
copy and paste terminal text just like Rxvt (with correct settings),
left mouse key select, middle button paste. It can even has transparency!
Truetype font, Multi-tab, customized tab names and settings... You name it.
It is just incredible. Now I run my python shell and ipython shell in
Console2. No more pain-in-the-arse cmd.exe copy/paste retardedness.
I hope this helps you, too.

Productivity Tool on Windows (For the GNU/Linux minded) -- First

I myself is a follower of open source software and also an addict of
GNU/Linux software tools. I cannot live without GNU tools, nor can I
program without OSS software. But in my current job, I am stuck with Windows
platform... Nevertheless, I gradually found my way to the old comfort
and joy all the GNU and other OSS software brought to me. Now, my Windows
development environment is good enough that I can live with all the short-
comings of Windows. I would like to share in my blog my experience. Hope
it will help all similar-minded others, too.

Foremost and most importantly, of course, is the shell. God knows why
Windows is still stuck with cmd.exe command line. That is really insanely
inconvenient. So a bash shell and a xterm like terminal window is of the
utmost importance.

I think most people will know the cygwin environment. Yes, that is also my
first choice. I will not talk about how to install cygwin here, as it is
so easy and www.cygwin.com is so easy to find. But I do want to point out
that, even some people install cygwin, they are still using the cmd.exe
terminal, which is default after cygwin installed. I would highly recommend
using the rxvt terminal of cygwin instead. As this make you copy and paste
from the command line so much easier than using the cmd.exe terminal. This is
easy to do. After cygwin installation, there will be a shortcut to the cygwin
lauch bat file on the Desktop (normally, unless you choose not to). Right-click
on it to bring up the context menu. Choose edit to edit the actual bat file's
content. Put something like this in it:

@echo off

REM C:
REM chdir C:\cygwin\bin

c:\cygwin\bin\rxvt --loginShell 

There you go. Rxvt and bash! See you stupid cmd.exe

Building wxWidgets on Windows

I just learned how to compile wxWidgets on Windows recently. Don't want to
forget about it. So just make a blog entry. May help others, too, if the
search engines rank it high enough...

wxWidget can be compile on Windows for different compiler tools. For
example, VC, MingW, borland C++, etc. I am only interested in VC and
MingW. If you are interested in commercial native tools, VC is sufficient
(just use the free beer version). If you are interested in OSS tools,
nothing is better than MingW. OK, that is my opinion anyway. I am very
glad to find out wxWidget support these two builds very easily and well.
Espeically the VC build can be just invoking the nmake tools. Hehe, go
to hell VC XML project files... The MingW build is simple enough, any
autoconf/automake user knows how to do it. Configure and make. Life is
so good!

I learned most of the knowledge from these two posts:
http://www.freelists.org/post/programmingblind/Tutorials-for-compilation-of-wxWidgets-on-Microsoft-windows
http://www.freelists.org/post/programmingblind/tutorials-for-compilation-of-wxWidgets-on-windows-part-2A-wxWidgets-and-MinGW

If there was ever a chance that these links are broken. Use a search engine
to search these keywords "compilation of wxWidgets on windows" (without quote).
A link can be broken, but Internet history will never die.

The only thing I think I should put down here is that, how to tweak various
settings of the build. wxWidgets has different build flavour, most importantly,
monolithic build or multi-lib build, unicode build or non-unicode build,
static lib build or DLL lib build, debug build or non-debug build, etc.
For VC build, these settings are tweaked through the config.vc file under
build/msw subdir of the wx Windows source tree. While for MingW build, the
settings are tweak on the configure script command line options. Running
./configure --help will show all the tweakable options.

After setting config.vc correctly, to build VC build, just need to invoke
nmake /f makefile.vc. Slick! For MingW build, run ./configure on top level
and then make. Nothing simpler!

OK. So much for wxWidgets building. Time to hack the code.

PS. Comiling wxWidget on Mac is no different than compiling it on Linux
or other UNIX system. Configure and make is the way to go.

PyQt vs wxPython

Recently, because of my job (I am a programmer working with C++, Python, GUI,
image processing, embedded system, Linux, Windows and Mac), I need to choose a
GUI framework for Python that are mature and cross-platform. Of course, the
best of the breed are wxPython and PyQt. But it seems to be hard to make a choice
between these two. The top search result of 'wxPython vs PyQt' is about 4 years
old. Qt/PyQt has gone through great changes. So is wxPython I believe. By
reading the old post doesn't seem to help me make a decision. I think the old
license issue (especially on Windows) is not very big an issue any more, since
Qt is open source (in the true sense) on Windows already. Yet, of course, the
GPL license is still scaring off quite a lot of people. I guess what
differentiate will be API beauty/neatness, documentation, support and maturity.
I used Qt before in the Qt2 days. I kind of think its API is neat. But some
post about PyQt pointed out that PyQt is not *Pythonic* enough. So I am not so
sure about the beauty of PyQt's API. From bits and pieces of reading, I think
now PyQt has improved a lot in this aspect, but I am not sure. If any one has
any experience, please leave a comment. Finally, I guess you don't really know,
until you try. Knowledge learn from other people's mouth is always not as good
as your own experience. I think I am going to give both PyQt and wxPython a
try.

I hear, I forget.
I see, I remember.
I do, I understand.