Monday 10 November 2014

Cassandra Handy commands



Some handy commands of Cassandra:

To make vertical output from cqlsh.

Use EXPAND ON;

Example :
cqlsh:loadtest> expand on;
Now printing expanded output
cqlsh:loadtest> select * from cmdtest;
@ Row 1
------+------
 id   | 1
 name | amit
(1 rows)



To execute Query from file

SOURCE

cqlsh> use loadtest;
cqlsh:loadtest> source '/tmp/query.sql'
 id | name
----+---------
  1 |    amit
  2 | testing
(2 rows)


CAPTURE :
Captures command output and appends it to a file.

To start capturing the output of a query, specify the path of the file relative to the current directory. Enclose the file name in single quotation marks

cqlsh:loadtest> capture '/tmp/a'
Now capturing query output to '/tmp/a'.
cqlsh:loadtest> select * from cmdtest;
cqlsh:loadtest> exit
cassandra@dell27:~> cat /tmp/a
 id | name
----+---------
  1 |    amit
  2 | testing
(2 rows)







Thursday 18 September 2014

Cassandra os-tuning.


OS TUNING - Cassandra

Below settings should be applied to all node of Cassandra cluster.


File Descriptors : Cassandra needs more file descriptor than default settings of 1024, below is how you can change it. 
#echo "* soft nofile 32768" | sudo tee -a /etc/security/limits.conf
#echo "* hard nofile 32768" | sudo tee -a /etc/security/limits.conf
#echo "root soft nofile 32768" | sudo tee -a /etc/security/limits.conf
#echo "root hard nofile 32768" | sudo tee -a /etc/security/limits.conf
User Resource Limits : 
Added the following entries in /etc/security/limits.conf file.
# End of file
* soft nofile 32768
* hard nofile 32768
root soft nofile 32768
root hard nofile 32768
* soft memlock unlimited
* hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited
* soft as unlimited
* hard as unlimited
root soft as unlimited
root hard as unlimited
Run command
#sysctl -w vm.max_map_count=131072
Disable Swap - Lets not jvm gets swapped rather get it killed by oom killer. As even swapped jvm eventually will killed by oom. Better node to down then slow. 
# sudo swapoff --all
Synchronize Clocks : The column will only get overwrite if recent version has latest time compare to old value. 
NTP is already running on server

Tuesday 16 September 2014

One liner linux



Linux -  makes life easy.. lets inspired with Linux



To see a line from big file.
         sed -n 'Xp' theFile, where X is your line number.
e.g          sed -n '42401p' /tmp/bd/soted_voter2.csv

To get more lines.
sed -n  '1127062,+10p' /tmp/gujrat/dhana_gujrat5

To remove a line from big file
sed -i '42401d' /tmp/bd/soted_voter2.csv
Note- If you dont put -i it will show whole output after replacement.

To get Non ascii/foreign characters of a file in red color.
grep --color='auto' -P -n "[\x80-\xFF]" /tmp/bd/soted_voter2.csv

To remove non-ascii chars from a file
tr -cd '\11\12\15\40-\176'  < /tmp/vr_votersorted2.copy > /tmp/bd/cleansortedvoter.copy
To grep pattern wise.
LANG=c;grep INSERT postgresql.log |grep 'Mar  10'|grep -P 'duration: \d\d\d' | cut -d' ' -f4 | cut -d':' -f1,2 | sort | uniq -c


To load data into cdb (only tab supported. )
cdbmake-12 jh jh.tmp < /tmp/jhepic1


To append some text at the end of line in a file.  
awk -F '`' '{print $1 "~2"}' epicmeta_stripped.csv > /tmp/jhepic


To move 1st 15 files from one dir to another.
find . maxdepth 1 -name '*voice*' | head -n 20 | xargs mv -t /tmp/p5


time perl -pi -e 's/0000-00-00 00:00:00//;' *2014*

To configure ip
ifconfig eth0 192.168.19.200/16 up


To Add gateway via command.
`route add default gw 192.168.2.234

To check which process is eating memory. 
ps -e -o pid,vsz,comm= | sort -n -k 2


Monday 15 September 2014

Cassandra-startups


Trying to brief Cassandra :  important stuff to learn to manage it. 



Cassandra -  In Greek mythology :   A Princess who can see the future but got cursed that nobody will believe on her.

Excellent nosql technology through which can you scale the writes and nail down all BCP and Disaster recovery issues.

Some of the important feature are listed below.

  • Multi Data center support
  • Multi master mode - peer to peer communication
  • High Availability
  • Linear scalability
  • Collection datatypes
  • Faster updates
  • Standard Monitoring Dashboard
Out of Scope
  •      joins
  •      group by
  •      order by
  •      secondary index - not performan
Important concepts
  • seed-node
  • Gossips
  • peer to peer
  • initial_token
  • listen_address
  • rpc_address
  • Compaction
  • tombstone
  • Quorum
  • consistency
  • replication factor
  • snitch

Need to learn:

Installation 

Connect to Cassandra (via cql)
create key space/table/index

How to add node
How to remove node
How to repair a cluster

How to backup c* cluster
How to setup monitoring  (OPS center)

How to set replication factor
How to set consistency Level (via Driver)
How to set authenticaion in connection