Run python program from notepad++

Open Notepad++ > Run... > Enter the following command and create a shortcut.


#cmd /k python.exe "$(CURRENT_DIRECTORY)\$(FILE_NAME)"
#cmd /k python.exe "$(FULL_CURRENT_PATH)"

Other variables that may be useful:
$(CURRENT_DIRECTORY)
$(FULL_CURRENT_PATH)
$(FILE_NAME)
$(NAME_PART)

Python : Check memory address of a variable

Python has a built in function to check memory location of a variable.
The variable "id" can be used for this purpose.

Using this special built in function you will get amazing insight on how assignment and other operations works in python.

eg: 

1. >>>A=1
2. >>> id(A)
3. 1403307696
4. >>> A=A+4
5. >>> id(A)
6. 1403307824
7. >>>

As you can see in python the memory address of a variable changes when we add integers. As integer variables are immutable similar to string. In this case when we add integer python allocated a new memory to variable and discarded older location.

On the other hand let's see what happens to a list:

1. >>> mylist=[1,2,3,4]
2. >>> id(mylist)
3. 841655129352
4. >>> mylist.append(5)
5. >>> id(mylist)
6. 841655129352
7. >>>

We can clearly see lists in python are mutable as the address of mylist variable remains same.

Python : List all imported modules

>>> print(help('modules'))

or

>>>help()
help>>> os
<.......>
This will list all methods/fuctions inside os module

or
>>import sys
>>print(sys.modules)
>>print(sys.path)
>>dir('__main__')

Python: Listing all functions , methods from a module using dir()

We can list all the methods or functions a python module provides.
You need to import the module first then use the dir() to list everything a module offers.
eg:
1. import os


2. dir(os)
>>> dir(os)
['DirEntry', 'F_OK', 'MutableMapping', 'O_APPEND', 'O_BINARY', 'O_CREAT', 'O_EXCL', 'O_NOINHERIT', 'O_RANDOM', 'O_RDONLY', 'O_RDWR', 'O_SEQUENTIAL', 'O_SHORT_LIVED', 'O_TEMPORARY', 'O_TEXT', 'O_TRUNC', 'O_WRONLY', 'P_DETACH', 'P_NOWAIT', 'P_NOWAITO', 'P_OVERLAY', 'P_WAIT', 'PathLike', 'R_OK', 'SEEK_CUR', 'SEEK_END', 'SEEK_SET', 'TMP_MAX', 'W_OK', 'X_OK', '_Environ', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_execvpe', '_exists', '_exit', '_fspath', '_get_exports_list', '_putenv', '_unsetenv', '_wrap_close', 'abc', 'abort', 'access', 'altsep', 'chdir', 'chmod', 'close', 'closerange', 'cpu_count', 'curdir', 'defpath', 'device_encoding', 'devnull', 'dup', 'dup2', 'environ', 'errno', 'error', 'execl', 'execle', 'execlp', 'execlpe', 'execv', 'execve', 'execvp', 'execvpe', 'extsep', 'fdopen', 'fsdecode', 'fsencode', 'fspath', 'fstat', 'fsync', 'ftruncate', 'get_exec_path', 'get_handle_inheritable', 'get_inheritable', 'get_terminal_size', 'getcwd', 'getcwdb', 'getenv', 'getlogin', 'getpid', 'getppid', 'isatty', 'kill', 'linesep', 'link', 'listdir', 'lseek', 'lstat', 'makedirs', 'mkdir', 'name', 'open', 'pardir', 'path', 'pathsep', 'pipe', 'popen', 'putenv', 'read', 'readlink', 'remove', 'removedirs', 'rename', 'renames', 'replace', 'rmdir', 'scandir', 'sep', 'set_handle_inheritable', 'set_inheritable', 'spawnl', 'spawnle', 'spawnv', 'spawnve', 'st', 'startfile', 'stat', 'stat_float_times', 'stat_result', 'statvfs_result', 'strerror', 'supports_bytes_environ', 'supports_dir_fd', 'supports_effective_ids', 'supports_fd', 'supports_follow_symlinks', 'symlink', 'sys', 'system', 'terminal_size', 'times', 'times_result', 'truncate', 'umask', 'uname_result', 'unlink', 'urandom', 'utime', 'waitpid', 'walk', 'write']


3. What a particular method/function does , we can get this help further by doing dir() as:
>>> dir(os.getcwd)
['__call__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__self__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__text_signature__']
>>>

>>> dir(os.getcwd())
['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'casefold', 'center', 'count', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'format_map', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit', 'isidentifier', 'islower', 'isnumeric', 'isprintable', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>>

4. get help()
>>> help(os.getcwd)
Help on built-in function getcwd in module nt:

getcwd()
    Return a unicode string representing the current working directory.

>>>

>>> help(os.rename)
Help on built-in function rename in module nt:

rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
    Rename a file or directory.
   
    If either src_dir_fd or dst_dir_fd is not None, it should be a file
      descriptor open to a directory, and the respective path string (src or dst)
      should be relative; the path will then be relative to that directory.
    src_dir_fd and dst_dir_fd, may not be implemented on your platform.
      If they are unavailable, using them will raise a NotImplementedError.

>>> 

Quick learn C to Assembly

I found this link to quickly learn or convert C to Assembly.

1. http://unixwiz.net/techtips/win32-callconv-asm.html

2. https://godbolt.org/

2. If you have source then with GCC convert to Assembly code:
# gcc -S hello.c

3. If you have only executable then use objdump : (uses executable )
# objdump -d hello.out

4. gdb: gdb list command

Signal vs Interrupt

Interrupts are for external events.
When an interrupt occurs , a context switch of already running thread is done to handle that interrupt.
Interrupts are handled by CPU for hardware events.
On receiving interrupt CPU does context switch i.e. saves the state of registers and starts processing the interrupt. (eg: keyboard input).

eg:
Keyboard input
I/O processing
high priority interrupt : NMI


Signals are more of internal mechanism to handle different events.
When a signal is sent to a process then OS interrupts the normal process flow control and handles that signal, if process has its own instructions to handle signal, else OS follows default signal handling. Signals are for process or threads and interrupts are for hardware/external events.

eg: SIGHUP, SIGTERM, 

Process vs Thread

Process states:
Running
Waiting:
Stopped
Zombie :Process died by its entries are still in the process table


Process has a virtual address space which is shared by threads it creates.

A running process has the following thread states :
Thread states:
Init
Ready/Runnable: TS_RUN
Running: TS_ONPROC
Sleeping: TS_SLEEP
Stopped: TS_STOPPED
zombie: TS_ZOMB

Threads inherit address space from parent process.
eg: fork. a child PID returns 0 if it is a thread.

stack and heap

Stack:
1. Allocated at compiled time
2. Fast access
3. eg: Arrays
4. LIFO (Last In First Out)
5. Difficult to modify/insert/delete
6. Can be sorted
7. Global variables declared in program, args passed to functions

Heap:
1. Dynamically allocated
2. slow to access
3. eg: malloc allocated memory as heap
4. Easy modify/insert/delete
5. Not Sorted
6. Difficult to traverse
7. Generally pointers used to access memory location

ref:
https://sites.google.com/site/wdhamilttutorials/c/q2/stack_vs_heap

Why use pointers in C/C++

Function return values: normal functions can return only single value. But on the other hand if we call function by reference then you can change multiple values without returning it.

Dynamic memory allocation:

Work directly on memory locations, avoid temporary variable assignment.

online compilers

List of online compilers:

https://ide.geeksforgeeks.org/index.php

https://www.tutorialspoint.com/compile_c_online.php

https://www.onlinegdb.com/
( for some reason recursive factorial in C returns 0, while other compilers works fine)

man vs apropos

man keyword shows the man page for given keyword

apropos keyword shows in which man pages this keyword appeared

Using translate "tr" to remove space or replace chars

Using translate (tr) to squeeze space :
# cat test
 Intel(R) Core(TM)   iii7-6600U CPU @      2.60GHz
 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
#

squeeze repeated spaces to a single space:
# cat test |tr -s ' '
 Intel(R) Core(TM) iii7-6600U CPU @ 2.60GHz
 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
#

Squeeze repeated 'i' to a single 'i' :
# cat test |tr -s 'i'
 Intel(R) Core(TM)   i7-6600U CPU @      2.60GHz
 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
#

squeeze all 'i' and replace to 'X' :
# cat test |tr -s 'i' 'X'
 Intel(R) Core(TM)   X7-6600U CPU @      2.60GHz
 Intel(R) Core(TM) X7-6600U CPU @ 2.60GHz
#

How to run shell commands in parallel under for loop

I came across a situation where i need to run multiple commands at the same time i.e. in parallel.
I had to run the for loop to capture the device name and then run some write tests on all these devices at the same time.

 A "for" loop is completed with "do" , "done" semantic.
 # for disk in `lsscsi |grep 'SDIFC10-0720801'|awk '{print $6}' `; do fio --ioengine=libaio --direct=1 --name=test --filename=$disk --bs=4k --iodepth=10 --size=1000M --readwrite=write ; done

But this will end up running the fio command sequentially one after another.

The little secret is to use "&" instead of ";" after do phrase :
 # for disk in `lsscsi |grep 'SDIFC10-0720801'|awk '{print $6}' `; do fio --ioengine=libaio --direct=1 --name=test --filename=$disk --bs=4k --iodepth=10 --size=1000M --readwrite=write & done

And this will run the fio command on all the disks at the same time.

Find a line with Control Character

I came into a situation where one of the application was failing to read a file.
There was no clue for that reason and it was just throwing exception of some bad string.
We knew there is something wrong with the file it is trying to read.

So we suspected some control character within the file. In order to find that character we can use the following commands :

Filename: test.txt
Line 1
line 2
line3
linne 4
line 5
line 6

GNU grep :
#/usr/gnu/bin/grep '[^[:print:]]' test.txt
line 5

#/usr/gnu/bin/grep '[[:cntrl:]]' test.txt
line 5

Above example indicates that there is a control character in the above line.

Using "cat" : reveals the control character at line 5 :
#cat -vte test.txt
Line 1$
line 2$
line3$
linne 4$
line 5^M$
line 6$

You can use vi/vim too but if the file is huge it may be difficult.
You can open file in vi and do ":set list" to see control characters

Print file in reverse using tac

This may not be very useful in everyday life but i just found this command that is available on Solaris as well as other *Nix platforms. If you want to print contents of a file you simply use command "cat" which will print lines from top to bottom. but if you want to print lines in opposite order then jsut use "tac".

Example :
~> cat test.txt
Line 1
line 2
line3
linne 4
line 5
----------
~> tac test.txt
line 5
linne 4
line3
line 2
Line 1
-----------------------------

As you can see in above example the lines are printed in reverse order.

Ubuntu LTS 14.06 Gnome

I was using beta for Ubuntu 14.04 Gnome for sometime on my virtual box. As i don't want to use Default Unity version as it is heavy on my Vbox. Hoping that these issues are fixed in final LTS release but unfortunately this is not the case.

1. But gnome had few issues as after boot it used to crash every time.

2. Sometime after login to gnome it comes up with just black screen and we have to reboot 2 or 3 times hoping that it will come up.
Workaround: Press Ctrl + Alt + F1 terminal will open login and do :
#startx
Otherwise : Enable gnome autologin from /etc/gdm/custom.conf

Automatically gnome session will start and you will get gnome login.

3. Ubuntu Software Center displays software description text which is almost invisible.





Hope these issues will be fixed in updates soon.

If you are interested in final releases for Ubuntu LTS 14.04 then you can download the same here :
Ubuntu Gnome: http://cdimage.ubuntu.com/ubuntu-gnome/releases/14.04/release/
Edubuntu : http://cdimage.ubuntu.com/edubuntu/releases/14.04/release/
Kubuntu: http://cdimage.ubuntu.com/kubuntu/releases/14.04/release/
Lubuntu: http://cdimage.ubuntu.com/lubuntu/releases/14.04/release/
Xubuntu: http://cdimage.ubuntu.com/xubuntu/releases/14.04/release/
Ubuntu Studio: http://cdimage.ubuntu.com/ubuntustudio/releases/14.04/release/

Dtrace : Time spent in system call or function

Dtrace is a awk and C++ like programming language also called D Language.
It can be used very efficiently to provide amazing results that other tools can't provide.
But Dtrace is not the replacement of other tools but you can say Dtrace is a complement to other tools.
Although it is not the first tool to be used while diagnosing a problem but it can be used at a later point of time to dig much deeper into the particular area of problem.

Let us take an example, if you are suspecting an application to be taking more time than expected then you may want to use first other tools to verify things like physical memory, CPU usage, swap etc. And then later we may want to start looking at the application.

Let us see if we just want to see which system calls or functions in an application are taking most time, then we can use the below simple script :

#!/usr/sbin/dtrace -s
pid$1:::entry
{
    t[probefunc] = timestamp;
}
pid$1:::return
/t[probefunc]/
{
    @funct[probefunc] = sum(timestamp - t[probefunc]);
    t[probefunc] = 0;
}

How to run this script ?
Save this script in a file eg. myprobe.d and give executable permissions. But note that ONLY root user or user with equivalent role can execute the script and provide process id "PID" of the application as an argument.
Now run the script as below and see the results :
# myprobe.d <PID>

In future posts I will explain what more about the Dtrace in details. This was just an introduction to see what Dtrace can do.

Process and open files

There are situations where we may need to know :
1. Which files are opened by a given process
2. Which process is using a given file

If we are given a process and we need to find out the files opened by it then we can simply use
#pfiles <pid>

And if a file is given an we need to know which process has opened it then use
#fuser <file>

Convert shell script in binary

There are times when you want to hide the contents of your shell script to binary form so that it may not be read by others.
One such utility is called "shcomp"

shcomp - compile a ksh93 shell script

Using this utility you can compile/convert your ksh script to a binary script.

Run command in Parallel

We can run shell commands in parallel which can be very useful to utilize the power of system
Today system come with multiple CPU and each CPU has multiple cores.
 To utilize the full power of these CPUs you can run shell commands in parallel of different CPU at the same time.

Below is just a small example shows dd command running in parallel using pipe "|" on different CPU :

 # dd if=dev/zero of=/dev/null | dd if=dev/zero of=/dev/null

 #prstat
PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
5080 root     1636K 1040K cpu0    40    0   0:00:24  17% dd/1
5079 root     1636K 1040K cpu3    40    0   0:00:24  17% dd/1

Similarly,
we can gunzip and tar a file :
# gzcat  <file.tar.gz> | tar -xvf -

# gzcat  <file.tar.gz> | tar -xvf -