Monday, May 22, 2017

Throwing in F25 into a machine running W10

Well, I did this today. I was on W10+Ubuntu 16.04 combo till now. Unfortunately for me, my Ubuntu was unaware of my HP240 hardware drivers. I went completely offline as both my wireless, ethernet, and BT were not recognized by Ubuntu. On top of that, I could not find cdc_ether drivers to tether using my mobile. After some struggle with trying different kernels, I decided to try F25. I used F25 media writer for windows. The first issue was that the link provided in Fedora download page does not have latest writer. https://getfedora.org/en/workstation/download/ points media writer version 4.0.8 which crashed before starting after installation. But, you get the latest from here and it will work (4.1.0 worked for me).

I used a 16 GB driver and after I booted to this drive, I installed fedora on my Ubuntu partition after deleting it. However, the installer cried out about not having GPT partition for boot loader. I just told it not to install a boot loader. I was hoping that I could get it installed after the OS is installed.

After OS was installed, I booted to HDD. It was stuck at grub. It will not proceed beyond that point as I have destroyed Ubuntu partitions and grub was looking for those partitions. All I got was a recovery shell of grub. I booted again to F25 USB stick installation. Did a chroot to newly created partition and tried almost all that was told here. No way. Error after error. My Sunday was getting spoiled. Just reminded me of those golden words about Sendmail....

I was getting disturbed. Decided to take a sure to work shortcut. Did the following.

  1. Went and bought another 32 GB pen drive
  2. Again booted to USB Linux image and downloaded Windows 10 dvd from Microsoft site and wrote it to a DVD. (No, I did not have to use this.)
  3. Took a back-up of MBR to pen drive
  4. Installed syslinux and recovered MBR (I was skeptical about effectiveness of doing this for W10 as it was suggested for every version of windows except Windows 10)
  5. After that step, my machine was able to boot to Windows (Hurray... a tiny step close to normalcy)
  6. Deleted the partitions in new 16 GB pen drive and also all Linux partitions from HDD
  7. Started the install to HDD process and selected both sda and sdc (pen drive) as target for installations
  8. Set pen drive as boot device and gave installer control over making partitions. It did a very good job by creating boot and EFI folders on pen driver and swap space and root on free space available in sda.
  9. Installation was complete and I removed all external drives and booted the device to Windows 
  10. Later I plugged in the pen drive and viola, and the device boots to Linux
I will take some time later and figure out how to avoid the pen drive. But till then, I have a working laptop that has both W10 and F25


Wednesday, March 29, 2017

Is JIO doctoring speed tests?

Well, it was only last week when one of my friend showed me the speed test app of jio reported over a few mbps as the download speed where as the app by TRAI showed the speed as 0.05mbps.

Today, I noticed something bizarre. I was watching youtube videos and suddenly speed went down and the videos started buffering. In a few seconds, i got a message from jio pay. I understood what happened and without checking the message, I opened the settings and changed mobile data to my vodafone sim. However, after a few minutes, i realized that it was close to 12am when it happened. So, at 12.03, I switched back to JIO only to find my videos still buffering. Now comes the fun part...

I opened the TRAI speed test app and checked the speed. It was showing 3.88 and 5.82 mbps for download and upload respectively. Something must be wrong. I checked youtube again. No change! It was buffering as before. I had half a mind to call up JIO guys and ask them why is youtube buffering at 3.88 mbps?

Anyway, before I did that, I opened the sms that i had got from JIO pay and checked its contents. Wow! It had news for me!

1. JIO does not restore your speeds at 12am. They do it at 2am
2. Till 2am my speed will be 128kbps

Now the mystery: If my speed was throttled at 128kbps, how did TRAI app read 3.88mbps? I believe this is a clear indication of the fact that JIO is doctoring the speed test results!

Friday, November 25, 2016

Streamer

This is some quick notes on cross compiling xawtv. I had to get a webcam working on an embedded device this week and after not getting much success with a simple v4l app, (derived from simplewebcam android app's native library), I decided to cross compile xawtv. The only interesting part was the streamer console application.

I cross compiled this for arm. There was not much support for cross compilation as far as I could see.

XawTv depended on libv4l for which I had to compile v4l utils. This in turn needed libjpeg support. SourceForge gave me version 6 of libjpeg which had issues with libtool while cross compiling. Eventually I built version 9 which was obtained from some other location. To get the library, i had to use --enable-shared flag at the time of configuring. Then I compiled v4l utils. All this was done by setting LDFLAGS, CFLAGS and CXXFLAGS appropriately. As I remember, compiling v4l utils needed -Wl,-rpath-link=path_to_jpeglib in compiled flags.

There was some code that needed x11 libraries and I just commented them using # if 0. Another point to note is that output goes to .lib for these packages.

Now, for xawtv, I set the compiler and linker flags and configured using --host option to indicate that it is a cross compilation. However, I ran into problems as LDFLAGS was not honoured. After a long struggle, I figured out that flag was called DLFLAGS or something similar in Makefile. I set that to my current LDFLAGS value. Things went fine to some extend and I got streamer compiled. However, I was getting many warnings and errors since the build kept looking at /usr/include though I was cross compiling. Using -nostdinc only worsened the pain as I had to then use -I flag for each standard include folder and sub folders. So, I gave up on that and looked at what was causing the error. I could see that among many warnings there was only one error. It was related to FD_XXX macros of select.

Though I first ignored all these and decided to use only streamer (which was already compiled at this stage), my happiness was short lives as I found that the streamer used libng and the plugins present inside that folder.

I then commented many places where select appeared. However, same was appearing in v4l drivers. There alone I replaced select with Poll. My streamer and drivers were now ready. But, while running, streamer was not able to load drivers due to some unresolved symbols in drivers.

After a feel in entire source code, I found that they were present in libng and I only had a static library for libng. At this point, a word about makefiles in xawtv. The top folder has main Makefile. It then had Subdir.mk in each sub-folder which is included by main make file. Quick look at Subdir.mk Will give one a good idea about how to build an application/library. After seeing the objects that go into libng.a, I wrote a simple make file to get a corresponding .so file (arm-xxx-gcc -fPIC -shared needed.o -o libng.so). Then I wrote a similar make file for streamer and used this so to build streamer. (Streamer was using .a earlier). Then I copied the streamer and new libng.so to target.

Viola! "./streamer -o image.ppm -D v4l2" gave me my first image and I ended my day's work after seeing it via ppm2tiff. Well, I was really refreshing to see the same old mundane scenes via that image :)

Wednesday, October 19, 2016

Android logging

I was looking at building a framework for logging which is expected to be platform independent. However, truth is that it cannot be really platform independent. If I say that all processes send the log to the logging framework using IPC and stop there, I will not be doing justice to the job of defining the architecture. I need to see what type of IPC is better for our QAs. (Here the main quality attribute that we consider is performance).

Now comes another problem. If I evaluate the IPC for one of the platform (custom made Linux) against IPC that in Android, (two of the five or six platforms that I need to consider) it might so happen that IPC mechanism chosen for Linux is not available under Android. So, I eventually decided to go bit deeper and look at the details of IPC mechanisms available in Android.

It was not that easy to understand the whole framework in a short duration. However, the following page was really helpful.

http://elinux.org/Android_Logging_System

So, every application uses a library function and writes to some buffers in kernel. I checked the corresponding code. Lets start with Log.java

The file is present in frameworks/base/core/java/android/util/Log.java
No magic seems to be happening there. It just calls corresponding native function println_native
println_native is in frameworks/base/core/jni/android_util_Log.cpp. However, the real work happens in __android_log_buf_write which is inside system/core/liblog/logd_write.c (or logd_write_kern.c depending on compile time flag). It calls write_to_log which is a function pointer which very intelligently implements a singleton.

Let us look at logd_write.c first:
At first write_to_log is set to __write_to_log_init which calls __write_to_log_initialize. Which opens a file "/dev/pmsg0" and a Unix socket /dev/scoket/logdw. The pointer is then modified to point to __write_to_log_daemon and subsequent calls reach that function.

In this function (__write_to_log_daemon), the log is written to both the socket and the device. We will first see what happens to socket.

The listener is in system/core/logd/LogListener.cpp. However, this class extends SocketListener. The real work is done in SocketListener::runListener. The socket is read for each client here. It took me a while to find out how the generic SocketListener class passes data to LogListener. SocketListener class has a virtual function onDataAvailable(SocketClient *c) which is implemented by LogListener. When the SocketListener finds that there is data to be read from the socket, it calls the onDataAvailable function  which maps to corresponding function of LogListener class.

What happens in LogListener::onDataAvailable?
The message is read and it is put into a LogBuffer. It also informs the reader when a new message arrives. All the connected clients can now get the data from the reader.

Apps-->Log-->/dev/socket/logdw-->LogListener-->LogBuffer-->LogReader-->/dev/socket/logdr-->Clients of log reader!

Well, I will update the log what happens to /dev/pmsg0 later. Time to sleep. Anyway, my takeaway from this is that I can safely use unix sockets for IPC in both Android and Linux without bothering much about performance and my code might be portable across these two platforms if I take a little bit of care.

Note: I have looked at Anroid 6.0.1 source to get this information.

Wednesday, September 21, 2016

Getting back to blogging

For a long time I was not blogging anything. However, recently I had to study Lucene for some project and I also started work on a application that is being developed on Linux. This made me think that I should also restart this blog. As part of the new application that I am working on, I have started learning python and I will post more about it later. I have lot of things planned for future. On the Linux front, I am looking at getting a test machine setup and spend sometime on some of the industry standard software (mainly from Apache software foundation).

I will take sometime, but, I will surely restart these blogs!