Search This Blog

Loading...

Monday, March 17, 2014

book read by Anton Roth in NBC crisis show

Yesterday (2014-03-16), I watched this show called crisis on NBC ( http://en.wikipedia.org/wiki/Crisis_(TV_series) ) . In that the kid who played Anton Roth reads a book titled "Passage to Perseid". Has anyone ever heard of this book? I searched in google, amazon but could not find it anywhere. Here is a screenshot of what I am talking about


Tuesday, January 14, 2014

perl one liner to print quotes around each line

Consider the sample input file
rajulocal@hogwarts:~/x$ cat input.txt 
1.1,k
2.2,ka
3.3,kam
4.4,kama
5.5,kamar
6.6,kamara
7.7,kamaraj
8.8,kamaraju
To put quotes around each line, use the following Perl one liner
rajulocal@hogwarts:~/x$ perl -lane '$sq="\047"; print "$sq$_$sq";' input.txt 
'1.1,k'
'2.2,ka'
'3.3,kam'
'4.4,kama'
'5.5,kamar'
'6.6,kamara'
'7.7,kamaraj'
'8.8,kamaraju'
Here we are using octal code of single quote. To put quotes around just one column, use
rajulocal@hogwarts:~/x$ perl -F',' -lane '$sq="\047"; print "$F[0],$sq$F[1]$sq";' input.txt 
1.1,'k'
2.2,'ka'
3.3,'kam'
4.4,'kama'
5.5,'kamar'
6.6,'kamara'
7.7,'kamaraj'
8.8,'kamaraju'

Related posts: awk remove quotes in a column

Tuesday, December 24, 2013

cvs equivalent of "svn status"


To get output similar to "svn status" while using cvs as the version control system, run
=> cvs -n -q update
U file1
U file2
M file3
M file4
? file5
? file6
This will display files modified upstream, files modified locally, files not tracked by cvs. The output is compact and similar to the output of "svn status".

Create an alias for this in the shell rc file
=> grep cvst ~/.cshrc
alias cvst    'cvs -n -q update'
Other Alternatives:
#!/bin/sh
#------------------------------------------------------------------------------
# Approach 1
# Initial version from : http://www.freshblurbs.com/blog/2009/02/08/cvs-status-one-svn-bash-script.html
# Problem: this will not display files not part of cvs.

patterns=( 
    '?' 
    'Locally Added'
    'Locally Modified' 
    'Needs Patch'
    )

for i in "${patterns[@]}"
do
  cvs -Q status -R . | grep -i "$i"
done
#------------------------------------------------------------------------------

#------------------------------------------------------------------------------
# Approach 2
# Initial version from : http://www.dzone.com/snippets/doing-something-equivalent-svn
# Problem: this will not display files not part of cvs.

cvs status 2>&1 | egrep "(^\? |Status: )" | grep -v Up-to-date
#------------------------------------------------------------------------------

Tuesday, September 17, 2013

file sizes of scanned images

When I scanned a passport (roughly 40 pages == 20 scanned images) in color with 600 dpi, the file size produced by Canon Pixma MX870 is 58 Mb. The same document when scanned using Xerox WorkCentre 7765 with same settings (color, 600 dpi) produces a file size of 2.3 Mb. In terms of speed, Pixma was like a grandma while the WorkCentre is very fast.

Up until now, I never fully appreciated the power of an industrial workhorse that costs around $34000 as opposed to a good personal inkjet printer ($130 on Amazon). Wow!

Tuesday, July 09, 2013

setenv does not work from script

This post explains how to export a variable from a shell script to a tcsh parent shell.
 
consider the following script
hogwarts:~/x> cat setup_var.sh
#! /bin/tcsh -f

setenv a RAJU
echo $a
The executable bit is set on the script
hogwarts:~/x> chmod +x setup_var.sh
hogwarts:~/x> ls -al setup_var.sh
-rwxr-xr-x 1 rajulocal rajulocal 39 Jul  9 03:40 setup_var.sh
 
I am using the tcsh shell
hogwarts:~/x> echo $0
tcsh
When executed, it shows that the variable "a" is getting the correct value
hogwarts:~/x> ./setup_var.sh
RAJU
However, the variable is not exported to the parent shell even though setenv is used in the script.
hogwarts:~/x> echo $a
a: Undefined variable.
The solution is to "source the script" instead of "dot executing".
hogwarts:~/x> source setup_var.sh
RAJU
hogwarts:~/x> echo $a
RAJU
Voila! Now the variable "a" is exported to the parent shell.

Happy shell script hacking...

Thursday, June 06, 2013

shell add multiple lines EOF

To create a file with multiple lines from a shell script you can use the following recipe
rajulocal@hogwarts:~/work/tcsh$ cat add_multiple_lines_to_file.sh
#! /bin/tcsh -fx

mkdir -p ~/x
cat << EOF > ~/x/junk1
k
am
ara
ju
EOF
Run this script using
rajulocal@hogwarts:~/work/tcsh$ ./add_multiple_lines_to_file.sh
mkdir -p /home/rajulocal/x
cat
The output will be stored in ~/x/junk1
rajulocal@hogwarts:~/work/tcsh$ cat ~/x/junk1
k
am
ara
ju
For further information
  1.  read the section on "Here-documents" in http://docstore.mik.ua/orelly/unix/ksh/ch07_01.htm
  2. http://en.wikipedia.org/wiki/Here_document
  3. search for "here document" in google

Thursday, February 28, 2013

generate random numbers with a given correlation in matlab

Q. How to generate random numbers with a pre specified correlation in Matlab?

Ans. Use mvnrnd() function.This function takes mean (vector of kx1), covariance (matrix of k x k), number of points (n). The output is an nxk matrix which corresponds to the multivariate normal distribution with the specified mean, covariance.
>> n=1000; mu=[-2,2]; sigma=[1 0.5; 0.5 1]; X = mvnrnd(mu, sigma, n);
>> size(X)
ans =
        1000           2

>> mean(X)
ans =
   -2.0350    2.0157

>> cov(X)
ans =
    0.9950    0.4898
    0.4898    0.9894

>> plot(X(:,1), X(:,2), '.')
>> grid

The problem is underspecified if only the correlation matrix is known. In this case, set the mean to a zero vector, covariance to the given correlation matrix.

Tested in MATLAB 7.14.0.739 (R2012a),  Octave 3.6.2

Thursday, February 21, 2013

append a number to a string in matlab

Problem: Given a prefix say 'foo', generate a cell array of strings in Matlab such that each string has a number appended to it.

Ans:
>> n=5; cas = cellstr([repmat('foo', n, 1), num2str([1:n]')])

cas = 

    'foo1'
    'foo2'
    'foo3'
    'foo4'
    'foo5'
 Spaces are automatically appended when the appended numbers have wider range.
>> n=10; cas = cellstr([repmat('foo', n, 1), num2str([1:n]')])

cas = 

    'foo 1'
    'foo 2'
    'foo 3'
    'foo 4'
    'foo 5'
    'foo 6'
    'foo 7'
    'foo 8'
    'foo 9'
    'foo10'
Adding another string at the top of the array is easy
>> n=10; cas = [{'raju'}; cellstr([repmat('foo', n, 1), num2str([1:n]')])]

cas = 

    'raju'
    'foo 1'
    'foo 2'
    'foo 3'
    'foo 4'
    'foo 5'
    'foo 6'
    'foo 7'
    'foo 8'
    'foo 9'
    'foo10'
which can also be done by
>> n=10; cas = cellstr([repmat('foo', n, 1), num2str([1:n]')]); cas2 = [{'raju'}; cas]

cas2 = 

    'raju'
    'foo 1'
    'foo 2'
    'foo 3'
    'foo 4'
    'foo 5'
    'foo 6'
    'foo 7'
    'foo 8'
    'foo 9'
    'foo10'

Thursday, October 11, 2012

do a grep on a column

Consider the following data
$ cat data.txt
1,fruit,apple red,spherical
2,fruit,apple green,spherical
3,vegetable,peppers green,irregular
4,vegetable,peppers yellow,irregular
5,vegetable,peppers red,irregular
6,vegetable,broccoli,irregular and green
7,plant,green spinach,leaves
8,plant,very green spinach,leaves
9,plant,verygreenspinach,leaves
10,seed,green pea,spherical
11,unknown,green,undefined
The problem is to filter the lines where the third field ends in the word green. So the output should be
2,fruit,apple green,spherical
3,vegetable,peppers green,irregular
Short answer:- use awk with regular expression support
$ awk -F"," '{if ($3 ~ /\sgreen$/) print $0}' data.txt
2,fruit,apple green,spherical
3,vegetable,peppers green,irregular
Long answer:-
Naive use of grep gives a lot of false positives.
$ grep green data.txt
2,fruit,apple green,spherical
3,vegetable,peppers green,irregular
6,vegetable,broccoli,irregular and green
7,plant,green spinach,leaves
8,plant,very green spinach,leaves
9,plant,verygreenspinach,leaves
10,seed,green pea,spherical
11,unknown,green,undefined
line6 should not be printed as the word green appears in the 4th column (and not the 3rd).

lines 7,8,10 has the word green in the 3rd field. But they should not be printed since green does not appear at the end.

line9 - the letters green are present in the third field but is not preceded by a space, so should not be printed.

line 11 is most likely a data error. The third field has the word green but is not associated with any fruit, vegetable, plant etc.,


Further, to print all the spherical objects, one can use
$ awk -F"," '{if ($4=="spherical") print $0}' data.txt
1,fruit,apple red,spherical
2,fruit,apple green,spherical
10,seed,green pea,spherical
Here a full match on the 4th field is performed. However, this trick cannot be extended to the present problem as only partially matches on the third field are desired.

How the solution works:-
$ awk -F"," '{if ($3 ~ /\sgreen$/) print $0}' data.txt 
~       tests for a match
/ ... /  delimiters of the regular expression
\s       test for space
$        test for end of field

Tested on Debian Wheezy using
$ awk --version
GNU Awk 4.0.1
Copyright (C) 1989, 1991-2012 Free Software Foundation.

Thursday, August 23, 2012

kill all the open figures in matlab

The following function can be used to kill all the open figures in Matlab

function kill_figures
  % close all open figures
  delete(findall(0,'Type','figure'));
end
This function comes in handy when doing parametric studies. Say, there is a big chunk of code that generates 10 figures per run. In order to study the effect of a parameter for say 5 cases, we end up generating 50 (=5x10) plots. Once the study is done, this function comes in handy to kill all those figure windows.

Related tips:

To get the handles of all the figures, do
figHandles = findall(0, 'Type', 'figure');


Sunday, June 24, 2012

specify rsync location

If rsync fails with the following error,
$ /usr/bin/rsync -prltvzD user@machine.com:~/file.csv . -n
rsync: Command not found.
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [Receiver=3.0.7]
the problem could be due to the non-standard location of rsync on the remote machine. For example, in this case, rsync was installed at /usr/local/bin/rsync on the remote machine. This location can be specified by using the "rsync-path" option. The following command succeeds.
$ /usr/bin/rsync -prltvzD --rsync-path=/usr/local/bin/rsync user@machine.com:~/file.csv . -n
receiving file list ... done

sent 20 bytes  received 58 bytes  3.47 bytes/sec
total size is 11473183  speedup is 147092.09 (DRY RUN)

Sunday, June 10, 2012

eBay should make the winning bid stick for at least an hour

Dear eBay[1],

In the current bidding process, the auction ends at a particular time which gives unnecessary importance to it. This can be evidenced by a lot of inactivity at the beginning and a frenzy of activity at the very end.

Eliminating this "auction-ending-time arbitrage" (AETA from now on) is very simple and easy. Just make each "winning bid" stick for at least an hour. If necessary, extend the "auction ending time" further.

You can be even more creative by providing this chance only to the previous "winning bidders". I will let your strategy team work out the finer details. But the underlying idea is very simple. You need to provide a better way for others to re-participate in the auction.

If you think I can be of further assistance, ping me kamaraju at gmail dot com.


Why is this such a big deal?

To be honest, the current system sucks. It is very easy to be outbid at the very last minute by a tiny amount.  There is not enough time left to reenter the bidding process when it happens. This can be very frustrating for the current winning bidder especially if he/she is leading the race for the last couple of days/hours. These winning bidders deserve a second chance to rebid and reparticipate in the auction.

This is not a big issue for items with a lot of bidders (high liquidity). But for items with a very few bidders (illiquid, niche items), the suggested improvement provides a good price discovery mechanism.

When you have a good price discovery platform, the sellers will be more inclined to use eBay resulting in more revenue. So it is a win-win for both eBay and its customers.

References:
  1. www.ebay.com
PS:- I have arbitrarily chosen the "sticky period" as one hour. But in reality, it can be any reasonable choice.

PPS:- If any one else agrees with me, go to http://pages.ebay.com/help/account/suggestions.html , click on the "Buying and searching" link, fill the form and make this suggestion to eBay.

Friday, May 25, 2012

iceweasel amazon prime flash error

When trying to watch a video on Amazon Prime, I got the following error
Sorry we were unable to stream this video. This is
likely because your flash player needs to be updated.
To fix the error, install the following packages
$dpkg -l \*flash\* \*iceweasel\* \*hal\* | grep ^ii
ii  flashplugin-nonfree                  1:2.8.3                              Adobe Flash Player - browser plugin
ii  flashplugin-nonfree-extrasound       0.0.svn2431-3                        Adobe Flash Player platform support library for Esound and OSS
ii  hal                                  0.5.14-8                             Hardware Abstraction Layer
ii  hal-info                             20091130-1                           Hardware Abstraction Layer - fdi files
ii  iceweasel                            10.0.4esr-2                          Web browser based on Firefox
ii  iceweasel-torbutton                  1.4.5.1-1                            transitional dummy package
ii  libhal-storage1                      0.5.14-8                             Hardware Abstraction Layer - shared library for storage devices
ii  libhal1                              0.5.14-8                             Hardware Abstraction Layer - shared library
ii  libkephal4                           4:4.4.5-9                            API for easier handling of multihead systems
ii  libkephal4abi1                       4:4.7.4-2                            API for easier handling of multihead systems
Note:- Some of the packages might be unnecessary. But they are sufficient.

Make sure that the hal daemon is working.
$ps aux | grep hald
116       5715  0.0  0.4  17108  4584 ?        Ssl  20:15   0:00 /usr/sbin/hald
root      5716  0.0  0.1  13980  1588 ?        Sl   20:15   0:00 hald-runner
root      5751  0.0  0.1   5864  1508 ?        S    20:15   0:00 hald-addon-input: Listening on /dev/input/event0 /dev/input/event2 /dev/input/event1 /dev/input/event6 /dev/input/event5 /dev/input/event3 /dev/input/event4 /dev/input/event10
root      5755  0.0  0.1   5860  1244 ?        S    20:15   0:00 /usr/lib/hal/hald-addon-rfkill-killswitch
root      5759  0.0  0.1   5856  1240 ?        S    20:15   0:00 /usr/lib/hal/hald-addon-generic-backlight
116       5777  0.0  0.1   3816  1176 ?        S    20:15   0:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
root      5778  0.0  0.1   5864  1508 ?        S    20:15   0:00 hald-addon-storage: polling /dev/sr0 (every 2 sec)
1000      5826  0.0  0.0   4048   764 pts/2    S+   20:25   0:00 grep hald

Update the flashplugin
$sudo update-flashplugin-nonfree --install --verbose

Close iceweasel window.  Remove the ~/.adobe , ~/.macromedia directories
mv ~/.adobe ~/.adobe_old_deleteafter
mv ~/.macromedia ~/.macromedia_old_deleteafter

Restart iceweasel. The video (and audio) should be working now.

Wednesday, May 23, 2012

awk remove quotes in a column

Consider the input file
$cat input.txt 
"k",1.1
"ka",2.2
"kam",3.3
"kama",4.4
"kamar",5.5
"kamara",6.6
"kamaraj",7.7
"kamaraju",8.8
Convert this to
$cat output.txt 
k|1.1
ka|2.2
kam|3.3
kama|4.4
kamar|5.5
kamara|6.6
kamaraj|7.7
kamaraju|8.8
i.e. remove the quotes in the first column, change the delimiter to '|'.

This can be achieved in awk by defining both '"' and ',' as delimiters.
$cat cmd.sh 
cat input.txt | awk -F "[\",]" -v OFS="|" '{print $2,$4}'
Execute the above script as
$./cmd.sh > output.txt 
Tested on Debian Wheezy using GNU awk 4.0.1 version.

keywords: awk remove quotes from string, get rid of quotes, replace the delimiter character

Sunday, April 08, 2012

who writes Linux

This is a great article my friend Sasikanth shared with me on facebook: who-writes-linux-2012 . It provides statistics on who actually develops the Linux kernel, the rate of development, companies supporting the developers and lot of other interesting tidbits.

It is good to see that most developers are getting paid for doing what they love. Makes it a more sustainable model.

The quote "As the Linux kernel grows, the rate of change is growing with it." just sums it up. This IMHO, is very hard to achieve with a conventional business model.

Followers