Sunday, August 31, 2008

The power of NC

Today my sister asked how to download songs from a particular site. (I am not revealing the details of the site here). When I checked the front page of the site, I found a note there saying I cannot download songs from there.
I decided to do a bit of work to find out what is happening. First I searched for an RTP capturing software. Then I thought to have a look at the data being transferred over the network. I used a network sniffer to get the stream. This is what I saw.

GET /$sitepreview/hidden.in/movie/medium/MovieName_OmKaram%2Emp3 HTTP/1.1^M
Host: 10.10.10.10^M
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1^M
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8^M
Accept-Language: en-gb,en;q=0.5^M
Accept-Encoding: gzip,deflate^M
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7^M
Keep-Alive: 300^M
Connection: keep-alive^M
Cookie: 1E1AC73C-EC31-4f8f-9A6C-305961899082=Hidden.in^M
Range: bytes=1699944-^M
If-Range: "0bf337a64b4c81:19434"^M
^M
HTTP/1.1 206 Partial Content^M
[402 bytes missing in capture file]^M

This was followed by some binary data. (I need to switch the browser. FF3 does something vague with the fonts. It pumps in Malayalam in the middle!). Too bad, konqueror does not even show me the compose window.
I first extracted the binary data (Just deleted the above shown lines) and played it. I could hear the song almost completely. But there were glitches. That must have been some part of application protocol. The notable point was that it was an HTTP stream and not a RTP stream.
That means, if I can pump in the text lines seen above (Till the blank ^M line just before "HTTP/1.1 206 Partial Content") and capture the raw binary bits that came from the server, that will be my song! Then I remembered about NetCat and it worked out perfectly well.
This is what I did.

1. Opened the webpage in the browser
2. Started the song by clicking it. It was being played by a flash player
3. Captured the initial few bytes of network traffic using my favourite sniffer.
4. Extracted the HTTP request from the message & saved in a file (s1.req)
5. cat s1.req | nc 80 > song.mp3

But then editing the captures were a pain. I found a way for that also. In every request, only the first line was different. So, I saved s1.req and for the rest of the files, just captured and saved first 3 to 5 seconds of data. Then I ran the following script.

#!/bin/bash
[ $# -eq 1 ] || echo "Usage $0 "
[ $# -eq 1 ] || exit 0
head -n1 $1.req > .req
tail -n10 s1.req >> .req
cat .req | nc 10.10.10.10 80 > $1.mp3

this will take the first line of the captured file, append it with the editied captured data and create a new request file for the current song.

I am currently planning to get the whole of that site on my laptop :)

Making of a channel

I need to answer 5 technical questions before I start making the channel

1. What does asterisk need from the channel? (What are the bare minimum features/functionalities of a channel?)
2. What all can asterisk take from the channel? (What is the largest possible set of features/functionalities that a channel can support out of the box?)
3. What are the features of the channel needed by the customer?
4. Is there any feature that a normal asterisk channel cannot support or it is very difficult to support?
5. How to add support to a new feature in channel?

I am searching the answer for the first question now.

Asterisk Channel

We have decided to make a new channel in Asterisk and almost completely move away from the zaptel channel. We had a discussion where we discussed how to proceed with the creation of the channel. In the discussion I was told that there is only a standard load_module function that is provided by Asterisk to define the channel. However, I was sure from my previous encounters with the channel that there needs to be something more than that and I even vaguely remembered that it is a structure.
Today I went back and had a look at the channel code and I found that channels infact use the structure ast_channel. I have not seen the code in detail. Will come back and post more once I see the channels in detail.

Saturday, August 30, 2008

Terminal emulation

For the past two weeks I was working on PRS terminal emulation. We have managed to get the emulation working to a large extend. Today, I helped the UTS guys with this program which wrote the status of their database sync to the terminal.
There were two issues they were facing.
1. Where to write?
The information has to be on screen when the PRS application runs. However, this can be running on any terminal. So, the programmer does not know if his application is going to run on tty1 or tty6, or even pts1 or pts25.
The solution was simple. I told him to write to /dev/tty. This is because, in Linux, /dev/tty always points to your current terminal. So, if I am in tty5 and do a echo 1 > /dev/tty, it will get echoed on my terminal (tty5). If I go to a virtual terminal like /dev/pts/2 and then do a echo 1 > /dev/tty, it will still appear on my current terminal, because /dev/tty will be pointing to /dev/pts/2!.

2. We need to write only one line that needs to be refreshed always. Other programs will be writing to same terminal but not on our line. How do I send the cursor back to where it was after I write my string?
Answer to this terminal sharing issue came from VT220 emulation commands. When you want to save the current cursor position and the characteristic, you just need to print ESC 7 ("\0337"). After that you can do whatever that you want with the termianl. Then you again write ESC 8 ("\0338") to the screen. It will restore all the settings that was present at the time of sending last ESC 7.

I am still working on a few other issues. Will post here once I find a solution.

Friday, August 29, 2008

MSP 455 XL & Serial port

I had to deal with this printer today at CRIS (Center for Railways Information Systems). We are doing POC (Proof Of Concept: A stage making the company eligible for taking part in CRIS tenders) for UTS (Un-reserved Ticketing System) and PRS (Passenger Reservation System) terminals at CRIS. What they want is to be able to run both applications simultaneously on the same thin client. However, both UTS and PRS need to print to line printers and the line printers of CRIS does not support USB. The solution was to use serial port for PRS and parallel port for UTS. MSP 455 has both ports and we decided to use it for testing.
However, we did not know the baud rates and the kind of cable to be used with the printer. We tried with a cross serial cable first. We tried all the bauds possible. (Baud setting: stty -F /dev/ttyS0 9600). Eventually I called up the TVS 'care line'. After a few hops, I reached a guy who told me that baud is 19200 and I need special TVS printer cables.
At this point I asked the hardware engineer with me to change the cable from cross to straight. He did that and then I started with 19200 and went all the way up to 1Mbps. Nope... only junk. Then I noticed that as I increase the speed, the data printed is getting lesser and lesser. => The speed need to be very low. I then tried 9600 and viola... It worked!
The lesson is MSP 455 from TVS needs a straight serial cable and it works at 9600 baud. We did simultaneous printing for UTS and PRS and it worked fine. Now I need to find a way to get the status of the serial printer. Will work on that tomorrow.

Friday, August 22, 2008

webmin

Had to modify a webmin module today and when I checked the page, the description was still the same. Then I remembered there was a cache.

>/etc/webmin/module.infos.cache

Access the page again and it is all fine.

Tuesday, August 19, 2008

DAHDI

As we know that zaptel is renamed to DAHDI, below are the links from which you can fetch DAHDI. I have not gone into the depths of DAHDI, but it looks like it is just a name change.

http://svn.digium.com/svn/dahdi/tools/trunk
http://svn.digium.com/svn/dahdi/linux/trunk

Wednesday, August 13, 2008

Training for Ankit (3)

The zap driver:

1. Used wcfxo.c as the datum

2. Created the following dummy functions
wcfxo_open
wcfxo_close
wcfxo_read
wcfxo_write
wcfxo_hooksig

3. Made the hello_spaninit function to initialize the span

4. Made the zaptel settings as I have shown in the first post in this series

5. Start zaptel service

6. Build and insert module

7. Run ztcfg

8. On the console, dial 1000

9. tail -f /var/log/messages should show message from hooksig

10. On the console, hangup

11. tail -f /var/log/messages should show message from hooksig

12. Since syslog buffers the message and does stuff like "Last message repeated X times" you might not see the second message immediately

#include
#include
#include "zaptel.h"

MODULE_LICENSE("GPL");

extern int gpltest;

struct hpvt {
int pos;
struct zt_span span;
struct zt_chan chan;
char variety[128];
}hello;

static int hello_spanInit();

static int hello_init(void)
{
printk(KERN_ALERT "Before: Hello, world %d \n",gpltest);
hello_spanInit();
printk(KERN_ALERT "After: Hello, world\n");
return 0;
}

static void hello_exit(void)
{
printk(KERN_ALERT "Before: Goodbye, cruel world\n");
zt_unregister(&hello.span);
printk(KERN_ALERT "After: Goodbye, cruel world\n");
}

static int wcfxo_open(struct zt_chan *c)
{

printk(KERN_ALERT "Open called\n");
return 0;
}

static int wcfxo_close(struct zt_chan *c)
{

printk(KERN_ALERT "Close called\n");
return 0;
}

static int wcfxo_hooksig(struct zt_chan *chan, zt_txsig_t txsig)
{

printk(KERN_ALERT "Hooksig called\n");
return 0;

}

static int wcfxo_watchdog(struct zt_span *span, int event)
{
printk("FXO: Restarting DMA\n");
return 0;
}


static int hello_spanInit()
{
struct hpvt *wc=&hello;

strcpy(hello.variety,"Thinvent");
hello.pos=0;
/* Zapata stuff */
sprintf(hello.span.name, "SARIN/%d", wc->pos);
snprintf(wc->span.desc, sizeof(wc->span.desc) - 1, "%s Board %d", wc->variety, wc->pos + 1);
sprintf(wc->chan.name, "SARIN/%d/%d", wc->pos, 0);
snprintf(wc->span.location, sizeof(wc->span.location) - 1,"HERE!");
wc->span.manufacturer = "SARIN";
strncpy(wc->span.devicetype, wc->variety, sizeof(wc->span.devicetype) - 1);
wc->chan.sigcap = ZT_SIG_FXSKS | ZT_SIG_FXSLS | ZT_SIG_SF;
wc->chan.chanpos = 1;
wc->span.chans = &wc->chan;
wc->span.channels = 1;
wc->span.hooksig = wcfxo_hooksig;
// wc->span.irq = wc->dev->irq;
wc->span.open = wcfxo_open;
wc->span.close = wcfxo_close;
wc->span.flags = ZT_FLAG_RBS;
wc->span.deflaw = ZT_LAW_MULAW;
wc->span.watchdog = wcfxo_watchdog;
#ifdef ENABLE_TASKLETS
tasklet_init(&wc->wcfxo_tlet, wcfxo_tasklet, (unsigned long)wc);
#endif
init_waitqueue_head(&wc->span.maintq);

wc->span.pvt = wc;
wc->chan.pvt = wc;
if (zt_register(&wc->span, 0)) {
printk("Unable to register span with zaptel\n");
return -1;
}
return 0;

}

module_init(hello_init);
module_exit(hello_exit);

Make file

obj-m += tzap.o
EXTRA_CFLAGS += -I/root/zaptel/kernel

all:
make -C /lib/modules/$(shell uname -r)/build M=$(shell pwd) modules

Tuesday, August 12, 2008

Training for Ankit (2)

Ok, we built the kernel module. What do we do with it now?

1. Insert the kernel module to kernel
insmod tzap.ko

2. See the messages appearing in /var/log/messages

At this point, Ankit asked me why tail /var/log/messages? Why can't we see that directly?
Ans: Insmod calls the hello_init function. The message comes from

printk(KERN_ALERT "Before: Hello, world %d \n",gpltest);

Now, KERN_ALERT is a priority for kernel messages. ALERT is not a high priority so that the message comes on our Terminal. We changed the line to

printk(KERN_EMERG "Before: Hello, world %d \n",gpltest);

This time, the message appeared on the Terminal. I then showed him /etc/syslog.conf and told him that, it is syslog which reads these messages and decides what to do with it. I showed him my /etc/syslog.conf and explained how various messages are destined to go to various locations.

3. Unloading the module

This is done with the command rmmod tzap. hello_exit is called when this happens.

4. Module license line.

MODULE_LICENSE("GPL"); or something like that tells the module is licensed under GPL license. This has an important for our customer. They will have to GPL the code if they plan to access the symbols that are exported using EXPORT_SYMBOL_GPL. Anyway, they are lucky and zaptel does not have GPL exports.

5. Init & Exit

module_init and module_exit defines the entry points for insmod and rmmod.

6. One more step __init and __exit

These macros tell the kernel that the functions thus defined are specific to module loading and unloading. I told him to remember the Linux boot process and the line "Freeing unused kernel memory". This is possible because if I mark a function as __init, the kernel knows it has no need of the function after initialization. This allows the kernel to free some memory after boot-up

Training for Ankit (1)

Today I was at customers development centre. Ankit is to develop the driver for the hardware they develop. This post will record the session that I had with Ankit.

Ankit was new to kernel programming. Even I have not mastered this beast well. Neverthless, I got Ankit started with it.

1. Zaptel discussion

It is not necessary to define a span in case of an FXS/FXO. Just need to define a channel.
However, for registering fxs/fxo also, we pass a span argument to zaptel. I asked Ankit to look at span as an abstraction for a device (E1 card, FXS etc) which 'n' channels. (n = 1, for FXS)
I showed him how to define a fxo channel.
zaptel.conf
loadzone = us
defaultzone = us
fxsks=1
zapata.conf (Additions only)
signalling=fxs_ks
callerid="Green Phone"<(256) 428-6121>
channel => 1
extensions.conf additions
exten => 1000,1,Dial(Zap/1)

2. Kernel module programming

To program kernel modules, you need to have kernel source for your current kernel.
I asked him to first download LDD3.
Then, we copied the code from second chapter and compiled the module.
This had to be done on my laptop, as he did not have kernel sources installed on his system.

#include
#include
MODULE_LICENSE("Dual BSD/GPL");
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);

Then we created the makefile

obj-m += tzap.o
EXTRA_CFLAGS += -I/root/zaptel/kernel

all:
make -C /lib/modules/$(shell uname -r)/build M=`pwd` modules

With the make file in place, we compiled the module and inserted it.


  1. obj-m is a variable understood by kernel build system
  2. obj-m = tzap.o indicates that the name of your module source file is tzap.c
  3. EXTRA_CFLAGS defines the extra compile time flags specific for your module
  4. In this case that extra flag is not really necessary. But as we do more work with this driver, we will need it.
  5. Make command arguments: -C Tells the make to change to before actually running
  6. Make command arguments: M=`pwd`defines M, which will be used by kernel build system to determine the directory in which module source is present
  7. Make command arguments: modules is the target to build
We invoke the kernel build system which in turn compiles the module for us.