Google fined USD 5 million for using patented code in Linux kernel

http://www.bbc.co.uk/news/technology-13168296

Very interesting news. I am still not really sure if this is part of mainline. However, considering the fact that the holder was able to analyse the code and ascertain that it was there in the Google's server's kernel indicates it was surely there on the public domain whether it was a custom piece of code written by Google engineers or not. This is surely something very interesting thing to happen and it would be very interesting to see if Google takes it beyond district court and what exactly happens to this case at the end

/. link: http://linux.slashdot.org/story/11/04/21/2140249/Google-Loses-Bedrock-Suit-All-Linux-May-Infringe

Kernel oops

I had to look into a kernel oops recently. This was produced on a ARM machine. It was relatively easy to understand this message. These messages have become much more simple to analyse these days. I am making this quick post to remember one register: LR

LR: This register holds the address to which PC will be set to once you hit return. This simply means, this is the caller of the function where the trouble occurred.

The point where failure happened can be seen by looking at PC. Also, SP points at the current stack location. When I scrolled down, I could see some good back tracing information too.

64 bit flash preview

Adobe is back with 64 bit flash for Linux. This is a preview and grab it before they close it again. Last time when they had tried this, there were a number of security issues and preview was pulled down soon.

Installation:

1. Download the preview version from Adobe labs site
2. Untar the downloaded plugin 
3. Copy the resulting plugin to your firefox plugin folder
1. I had to first create this: mkdir /usr/lib64/firefox-3.6/plugins
2. Copy libflashplayer.so to that folder
4. Just re-start firefox and you are ready to browse the flash sites :)

Download link
http://labs.adobe.com/downloads/flashplayer10_square.html

(Its not easy to find the link. It took me 10 minutes to find this. Though there is a get beta note on the download page, it does not take you directly here)

Silence the console

Well, at times, its very much annoying to hear the console beep from a Linux terminal. Sometimes, it is very untimely. For example, you might be stuck at some command and might press a tab. If there is no possible completion, terminal beeps. Its more like your leg is stuck between tracks when the train is approaching and you are desperately trying to escape. Then the guard standing near you blows his whistle and asks you to get of from the track fast, without trying to help you. Anyway, Here is how you disable it

For text terminals

setterm -blength 0

For graphical terminals

xset b off

And u have peace....

SCU on ARM11MPCore will not give you coherency

Well, we thought otherwise. But, that was not correct. This was a very interesting experience. We started writing the communication module to make two different cores of ARM11MPCore communicate and found that things are not going fine as we were expecting them to. We write from one core and we cannot read it back from other. First suspect was the new virtual addresses that had happened in VxWorks. However, that was not the issue. After reading about it a bit, we were convinced that it must have been a linear mapping in our case. Now, what was the reason? The next suspect was the cache. However, the process was not simple. No one except me in the whole group believed that it can be caused by cache. Also, I myself was doubtful as I felt SCU should take care of it.

To clear our doubt, we talked to a manager in our company who was an expert in JTAG. He appeared completely assured of the fact that we are doing something dumb with JTAG as we cannot see this memory even in JTAG. When I asked about cache coherency issues, he brushed aside my doubts saying that it can never happen.

Well,  after giving the advice, he walked away without turning back. But, I was the tech lead for this project and I had to get this working somehow. And, something kept telling me that this is a cache issue. So, I decided to keep hammering at that point. Now, we were writing from VxWorks and reading from Linux. I decided to look for code which will flush cache. But, being a noob in VxWorks, I could not find this code. We tried some silly tricks in the beginning like increasing the memory size in the hope that cache will get flushed auto-magically. Fortunately or unfortunately, these tricks did not work. However, one guy in our team was able to dig out VxWorks function to flush cache. It was a simple function that took the cache type, address and length as the arguments and it did its job! (Yes that was simple compared to what I found for Linux. We had to get physical address of the memory, get page structure and then flush the page from cache!) Yes, after flushing the cache, we were able to see what was being written by the second core into the memory location from the code that was running in first core. Later, another member in my team found that SCU will not do this in case of ARM11MPCore. It does this in case of some other newer chips (Cortex-A9). We also read that allocating DMA memory might do the flushing directly. We are yet to try this.

Linux IPI handling for ARM11MPCore (ASMP)

We have been struggling for a while with this. Last Friday, I nailed him down.Ok, the background story...

We knew that IPI (Inter Processor Interrupt) can be used to communicate with other processor. Linux has support for this in SMP mode. However, we are doing ASMP. We wanted to use it slightly differently to device a custom communication module with another core running VxWorks. Now, the first hurdle came in the form of not being able to write to some registers. That was a dumb mistake of not ioremapping the register addresses. After that we faced another problem of not being able to register an interrupt. After a bit of debugging, we found that the reason is that there is no chip descriptor registered for interrupts 1-16 (They are the interrupt numbers for IPI). Well, the init code for interrupt was doing it for interrupts >= 29. But why?

It is this question that forced us to spend a week to resolve. There was one guy working on it continuously and I will break my head whenever I get time. Now, we tried the most obvious. We changed 29 to 1 in init code for interrupts. This enabled us to register the interrupt. However, this did not cause our interrupt function to be called when we wrote to the register. Last Friday, I decided to look at realview code. And it paid off. I could see that, they are also registering the interrupt from 29 onwards in the initialization code. Now, what was the problem? To know that, I had to delve much deeper. The answer came from the low level vector interface routines. The do_IRQ was not responsible for handling IPI. Instead, there was a do_IPI function. We immediately reverted the changes done in the irq initialization code. Then, we put some code into do_IPI

Well, thats it! our interrupts started working. We did face some issues while writing to the shared memory. All about that in another post.

Process holding the block devices

Recently I had to write a program to shred a live disk. This was a security requirement for our devices. However, while testing the program, I found that this always failed. It was possible to overwrite whole of the disk except for a single sector. I knew that some process is using this part of the disk and that is the reason I was not able to write to it. I did an lsof for the device when the write operation was going on. There were certain process having files opened from that device. However, even after killing them, my program was not successful in writing the entire disk. Later, I even figured out that the programs with open files were not causing any real problem! But, then what was blocking the overwrite?

Now, we have decided to reboot the machine and do this operation before boot up. However, my doubt stayed till today. It is only today that I came to know about the real culprit. It is none other than the kernel. When a FS is mounted, the disk is opened by kernel. As long as it is mounted, kernel holds this open fd. As mount would have exited long back, so, one will never see this being caused by mount!

Looking for Linux kernel and C programmers (jobs)

Ok, we have been paining our HR for a while for this. But, day by day I am losing my confidence on getting any good resumes from them. We are looking for people with C and Linux internal skills. We have three openings. The first one is for Linux system programmers. The second one is for a device driver expert (PCIex, PCI, USB...). Last but not least, we are also looking for someone good at Linux (and optionally VxWorks) BSP. Knowledge of uboot will be an added advantage. If there is someone who is good at all these you are very much welcome. Also, it goes without saying that you should be very good at C.

If you have some extra Java or C++ skills, its going to be and added plus point. But, they are strictly optional. Btw, though we are mostly known for our memory chips and mobile phones, this opening is for our printer division. If you are interested, please send in your to suseelan at gmail.com

How (not) to get chrome OS running in vmware

This weekend, I was just trying out Google's Chromium OS. The build went fine, all credits to Gentoo. But, the script to make a vmware image out of it failed miserably. After a lot of debugging I realised that a post install script is getting called which is causing all these issues. Now, what does the post install do?

/postinst messes up with your disk! Thank god that I realised this and I did not have something called /dev/sda12. Whatever be the the reason, they should have at least warned the users that image_to_vmare.sh script will try to mess up with the disk. Since I was pretty sure that I am interested in a vmware image, and no disk need to be involved here, I simply commented out that and proceed.

Once the image was formed, it was not booting in VMWare. Then I downloaded a openfiler vmware image and mounted chromeos disk as secondary. After a bit of tweaking with the grub configuration file (/boot/extlinux.conf), I was able to see something on the screen. I did the following things for this.

1. Marked /dev/sdb12 as a bootable partition using cgpt
2. Removed quiet and console options from kernel command line
3. Changed from USB boot to hdd boot
4. Changed the root device to /dev/sda3 and changed the ro option to rw

Now, comes the second pain. The X won't startup. I still haven't got it solved. But, I can currently run chrome. Lets see what I did for that.

1. Create a back-door entry by editing /etc/init/tty2.conf
1.1 Ask agetty to use /bin/bash as login program
2. Create a small shell script in / and pass the path to it as user name
(These were again done by mounting the disk on another working vmware appliance)
3. Create a /etc/init/network.conf for bringing up the wired network (Yep, that also did not work. Added a modprobe, ifconfig and route to do this)

Booted the machine and after ui.conf failed, I switched to tty2. Just gave my script path as login name and I got a shell. Now, I was able to start the X server. Created a new config file using X -configure and copied it to /etc/X11/xorg.conf. After this, I put the following line to ~/.xinitrc

echo /opt/google/chrome/chrome > ~/.xinitrc

Now, I typed in xinit and I have the chrome running!

There were many other painful stuff that I had to go through from yesterday morning till today 11AM to get this stuff working. But, I am still not happy. I want the UI to come up auto magically and let me login to google. I can see the reboot speed is really fast. I say reboot and the its done in 2-3 seconds! But, I really want to see the time taken by the actual UI to come up. If I get this working, I will post it here.

Currently, chrome allows me to browse the web like it does on a normal browser. Only change I could see was the presence of network icons on the top right corner. Also, if I directly type in http://google.com, it crashes. It seems like it is looking for some key manager and does not find it. At times, I can see it cribbing about CrOS library not getting loaded. Anyway, I only spent a day on this. Rest, next week :)

FreeMind

Well, I just read about this in LFY today and wanted to try it. Seems like its a cool tool. I made a introductory Linux presentation (meant for our Java developers) in this. May be it is not a real strong use case. But, Freemind sure did help me organize things a bit. The software has got some nice folding and re-arranging options which I liked.

You can find FreeMind at http://freemind.sourceforge.net/wiki/index.php/Main_Page

I am also uploading the stuff that I made (exported to jpeg) using this.

Kernel Thread

This post is on how to create a simple kernel thread. Note that the idea here is to explain the kthread and hence not much importance is given to other aspects.

---

kthread.c

---

#include <linux/module.h>
#include <linux/sched.h>
#include <linux/kthread.h>

struct task_struct *ts;

int my_thread(void *data)
{


    printk("%s\n",data);<
    while(1)
    {
        printk("I am a kernel thread\n");
        msleep(3000);
        if(kthread_should_stop()) return 0;
    }
    return -1;
}

int tinit(void)
{
    ts=kthread_run(my_thread,"message from init","MyKernelThread");
    return 0;
}

void tstop(void)
{

    kthread_stop(ts);
}

module_init(tinit);
module_exit(tstop);

---

Makefile

---

obj-m := kthread.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=`pwd` modules


clean:
    @rm -f *.ko *.o *.mod.o *.mod.c



How does it work?

It is very simple. The function kthread_run is not actually a function. It is a macro defined in include/linux/kthread.h. This macro takes care of creating and starting the kernel thread. my_thread is the function that will be run by this thread. All these are very similar to user land threads. However, there is a notable difference here. From the thread function you should keep checking if your thread is being stopped and return. This is done by calling kthread_should_stop(). If this function returns true, that means it is time for us to return. This is a nice feature as this will allow us to do some cleanup before we exit. (This is a must since, if this function does not return, your rmmod will hang). As you can see, the thread is stopped by kthread_stop call. So, what are you waiting for? Have some fun with your brand new kernel thread

syscalls

man 2 syscalls

This man page lists all system calls. Its a nice experience to view this page and then go through each of the system call man page.

Hmm... This reminds me that I am still running 2.6.23. Time to upgrade and have a new kernel and man page.

Making of a cd image

Last week, I wanted to write a few files to a cd and give to my father. Those were mp3 files. I tried mkisofs (genisoimage) but the result was not acceptable. Almost all file names were screwed due to name size limitation of iso9660. Also, I did not want the folder hierarchy as I was not sure if my dad's mp3 player will accept it.

Today I sat down to make this happen. There might have been other ways. But, this looked simpler than googling for hours for the most optimum ways....

Write a small program to do the linking...


#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[])
{
    int i=0;
    char str[128];


    if(argc>1)
    {
        for(i=1;i<argc;i++)
        {
            memset(str,0,sizeof(str));
            sprintf(str,"ln -s \"%s\" .",argv[i]);
            printf("%s\n",str);
            system(str);

        }
    }

    return 0;
}

Compile it into a program (say lnk)
gcc lnk.c -o lnk

Create a folder to hold the links to your mp3 files & enter that folder
mkdir temp
cd temp

Create the link from source folder to here
find ../mp3-collection/ -name "*.mp3" | xargs -d '\n' ./lnk

Make your iso file...
cd ..
genisoimage -ldots -allow-lowercase -allow-multidot -f -J -joliet-long -r -o temp.iso temp/*

(I spent 5 - 10 minutes reading genisoimage man page. Feel free to do the same. -f: follow sym link, -J: Joilet extn, -r: Rock ridge extn)

you can now mount this and check the file names etc (mount -o loop temp.iso /mnt). After you are happy with your brand new iso image, use wodim to write it to a cd

wodim dev=/dev/cdrw temp.iso

Catch: We are creating sym links from a folder tree to a single folder. This can cause issues with same file name repeating. An enhancement to lnk.c can be to check the return status of 'system' call and then modify the file name string accordingly.

clearcase find

Today I had to merge main branch files to a private branch. The reason was that people was updating the main branch and private branch was having many issues. However, when it came to updating files in private branch, I could not find which are the files that were branched out. This forced me to refresh some old commands of clearcase. I used a simple find to get a list of all files which were on a private branch.

cleartool find ./ -element "brtype(Private_branch)" -print

Let me also post some sites from where you can get some good information on clearcase commands.

1. http://www.yolinux.com/TUTORIALS/ClearcaseCommands.html (Has some simple examples)

2. http://publib.boulder.ibm.com/infocenter/cchelp/v7r0m0/index.jsp?topic=/com.ibm.rational.clearcase.hlp.doc/cc_main/c_hood_cleartool_cli.htm

3. Last but not least, use cleartool man to get offline help. (I goes without saying that you should have cc installation if you have to use it)

mysql-embedded

Well, I was looking at sqlite last week. But while doing a yum update I found this rpm package. "mysql-embedded". On doing a rpm -qi, it told me that this is a C interface to mysql that can directly be embedded in your application. I haven't tried it yet. But I am sure its going to be interesting. The package has only one shared library. However the size is ~7.5M. If I happen to use it, will post again here.

Quick popup

Today I was downloading a file using bittorrent. I was using my laptop. I kept it on my lap and started working on a document when download was going on. After a while I noticed that the download has stopped. When I checked, I found that the network cable had come out. This repeated for a while and I found that how much ever I try, the cable will come out slowly as it was not really fitting in the socket. By the time I notice this, 10-15 minutes would have elapsed without network.

How do I know just when it gets disconnected? I was running XFCE. I knew that XFCE allowed to create widgets that can do this for me. But, reading and understanding them and again writing one will take time. Also, I cannot use any built in one as I want a real popup. (My panel auto hides. Can't waste height on a wide screen monitor).

This is what I did.

while sleep 1 ;
do
ping -c 3 www.google.com ;
[ $? == "1" ] && xterm -e "wall trouble && sleep 10";
done

Thats all! When my network went down, xterm just screamed on my face that there is trouble. Then I will push the cable a bit and press ctrl-c to dismiss the message. I love xterm... :)

/dev/port

This is something that I had known for years but never had a chance to try out. Recently I had to write a program to control parallel ports. I decided to do with iopl and in/outb. However, they did not work for me. Then I tried this. Just open the file, seek till where your parallel port is, write the data! It worked like a charm. Below is a small code snippet to do it.


#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(int argc, char *argv[])
{
    int fd1=-1;
    unsigned char data=0xaa;

    fd1=open("/dev/port",O_RDWR|O_NDELAY);
    if ( fd1 < 0 ) return fd1;

    lseek(fd1,(0x378+0x00),SEEK_SET);
    write(fd1,&data,1);


    return 0;
}



You will have to change 0x378 to your parallel port base address. Also data is at offset 0, status is at offset 1 and control is at offset 2.

MAXNS

We recently had a requirement to have about 8 entries in /etc/resolv.conf. This was because the machine was to be used in various networks where the NS entry will be different. However, the default value of MAXNS in resolv.h is only 3 (There might be distros that might have set this to another value. But most of them have it at 3). That means, if I add a 4th or 5th name server to /etc/resolv.conf, all of them (starting from 4th entry) will get ignored.

What is the workaround? There are two ways to solve this. The first solution is to change MAXNS to some other value in resolv.h and recompile the stuff. Now, what is this 'stuff'? This stuff is glibc. Replacing glibc on a working machine can be disastrous. Hence I decided to recompile glibc on a new machine. Again, what if the machine did not accept a newly compiled glibc? I might not lose any data, but the effort spent will be wasted. Hence, I decided to make an lfs.

I downloaded an lfs live CD from here . Its been ages since I had done it last. This time, however, I decided to use jhalfs. LFS live CD has a user with the same name and the tool is present there. The glibc source tar ball was replaced by a modified one where the MAXNS was set to 8. It took one whole night. But it worked. I was able to set 8 entries in resolv.conf and they were all used.

What is the second solution? It is much simpler. Just install your own DNS server software and configure it in proxy mode. I did not try this. But, I am almost sure that it should work. Alas, its just a small piece of software. You can change it in anyway to accept any number of i/p and if you want you can write one overnight. We are evaluating both the solutions. They both have some common issues.

1. Resolution will take a long time if the address need to be retrieved from name server at the bottom of the list
2. If any one of the top servers decides to respond for the query with an address not found, then even if we have a correct answer from the bottom server, we will not get it.

However, the second solution has some workarounds for these two issues. The above two problems can be solved with the help of domain specific forwarders (named supports this AFAIK).

Soft realtime

Recently we wanted to try out some hard real time stuff. When we attempted it, the results were not satisfactory. Today, I just tried a program to see how a soft real time program performs against a normal program.
(Notice the use of SCHED_FIFO and a priority

#include <stdio.h> #include <sched.h> #include <sys/time.h> int main() { int retval=-1; struct sched_param sp; struct timeval tv; int old=0; #if defined(SFT_RLT) sp.sched_priority=20; retval=sched_setscheduler(0,SCHED_FIFO,&sp); printf("Ret= %d\n",retval); #endif while(1) { gettimeofday(&tv,NULL); old=(tv.tv_usec-old)<0?1000*1000+tv.tv_usec-old:tv.tv_usec-old; printf("%u %u %u \n",tv.tv_sec, tv.tv_usec,old); old=tv.tv_usec; usleep(1000*200); } return 0; }

I compiled two binaries from this

gcc -DSFT_RLT rt.c -o rt
gcc rt.c -o nrt

As one can easily see, the first one is a soft realtime program and the second is a normal program. I ran these programs under two scenarios. One, when the system was relatively free; two, when the system was loaded. This is what I did and observed.

Normal scenario (X, FVWM, Terminal, bittorrent and Firefox)

./rt	./nrt
1246142672 137029 200069 1246142672 337101 200072 1246142672 537171 200070 1246142672 737242 200071 1246142672 937312 200070 1246142673 137344 200032 1246142673 337413 200069 1246142673 537486 200073 1246142673 737557 200071 1246142673 937627 200070	1246142672 004983 200072 1246142672 205055 200072 1246142672 405127 200072 1246142672 605199 200072 1246142672 805265 200066 1246142673 005331 200066 1246142673 205402 200071 1246142673 405538 200136 1246142673 605604 200066 1246142673 805668 200064

Then I tried running the same under loaded condition. (Just played a movie using mplayer. The other processes which were earlier running were running now also)

./rt	./nrt
1246146025 832916 200046 1246146026 032951 200035 1246146026 233010 200059 1246146026 433064 200054 1246146026 633097 200033 1246146026 833150 200053 1246146027 033203 200053 1246146027 233255 200052 1246146027 433286 200031 1246146027 633317 200031 1246146027 833368 200051	1246146025 803540 200145 1246146026 003647 200107 1246146026 203734 200087 1246146026 403823 200089 1246146026 603918 200095 1246146026 804006 200088 1246146027 004127 200121 1246146027 204204 200077 1246146027 404257 200053 1246146027 604314 200057 1246146027 804368 200054

I guess the difference in result is noticeable.This is easier to achieve compared to hard rt, since there is no kernel patches needed. When we make a break through with hard rt, I will post again

Analysing a file (brute force method)

Recently I came across a page where someone was asking about finding the offset of a file system in a disk image. I am trying the following. Will update this post if I succeed.

for k in `count 0 1 100` ; do for d in `count ${k}000 1 `; do dd if=working.img of=ram/test.img skip=$d bs=1 count=512 &> /dev/null; file ram/test.img | grep ram/test.img | grep -v data; [ $? == "0" ] && echo $d;done ;done

count : this is a small program that I wrote. It take 2 or 3 parameters. "count 0 1 100" will print "0 1 2 3 4... 100". "count n 1" will print "n n+1 n+2.... n+1000"

The directory ram was mount point for a ram disk. It was created as follows

mkfs -t ext2 /dev/ram4
mount -o loop /dev/ram4 ram

Still its taking a lot of time and its eating a lot of cpu. Next, I will move the working.img also to ramdisk!