Cassandra os-tuning.

OS TUNING - Cassandra

Below settings should be applied to all node of Cassandra cluster.

File Descriptors : Cassandra needs more file descriptor than default settings of 1024, below is how you can change it.

#echo "* soft nofile 32768" | sudo tee -a /etc/security/limits.conf

#echo "* hard nofile 32768" | sudo tee -a /etc/security/limits.conf

#echo "root soft nofile 32768" | sudo tee -a /etc/security/limits.conf

#echo "root hard nofile 32768" | sudo tee -a /etc/security/limits.conf

User Resource Limits :

Added the following entries in /etc/security/limits.conf file.

# End of file

* soft nofile 32768

* hard nofile 32768

root soft nofile 32768

root hard nofile 32768

* soft memlock unlimited

* hard memlock unlimited

root soft memlock unlimited

root hard memlock unlimited

* soft as unlimited

* hard as unlimited

root soft as unlimited

root hard as unlimited

Run command

#sysctl -w vm.max_map_count=131072

Disable Swap - Lets not jvm gets swapped rather get it killed by oom killer. As even swapped jvm eventually will killed by oom. Better node to down then slow.

# sudo swapoff --all

Synchronize Clocks : The column will only get overwrite if recent version has latest time compare to old value.

NTP is already running on server

One liner linux

Linux -  makes life easy.. lets inspired with Linux

To see a line from big file.

         sed -n 'Xp' theFile, where X is your line number.

e.g          sed -n '42401p' /tmp/bd/soted_voter2.csv

To get more lines.

sed -n  '1127062,+10p' /tmp/gujrat/dhana_gujrat5

To remove a line from big file

sed -i '42401d' /tmp/bd/soted_voter2.csv

Note- If you dont put -i it will show whole output after replacement.

To get Non ascii/foreign characters of a file in red color.

grep --color='auto' -P -n "[\x80-\xFF]" /tmp/bd/soted_voter2.csv

To remove non-ascii chars from a file

tr -cd '\11\12\15\40-\176'  < /tmp/vr_votersorted2.copy > /tmp/bd/cleansortedvoter.copy

To grep pattern wise.

LANG=c;grep INSERT postgresql.log |grep 'Mar  10'|grep -P 'duration: \d\d\d' | cut -d' ' -f4 | cut -d':' -f1,2 | sort | uniq -c

To load data into cdb (only tab supported. )

cdbmake-12 jh jh.tmp < /tmp/jhepic1

To append some text at the end of line in a file.  

awk -F '`' '{print $1 "~2"}' epicmeta_stripped.csv > /tmp/jhepic

To move 1st 15 files from one dir to another.

find . maxdepth 1 -name '*voice*' | head -n 20 | xargs mv -t /tmp/p5

time perl -pi -e 's/0000-00-00 00:00:00//;' *2014*

To configure ip

ifconfig eth0 192.168.19.200/16 up

To Add gateway via command.

`route add default gw 192.168.2.234

To check which process is eating memory. 
ps -e -o pid,vsz,comm= | sort -n -k 2

Cassandra-startups

Trying to brief Cassandra :  important stuff to learn to manage it. 



Cassandra -  In Greek mythology :   A Princess who can see the future but got cursed that nobody will believe on her.

Excellent nosql technology through which can you scale the writes and nail down all BCP and Disaster recovery issues.

Some of the important feature are listed below.

• Multi Data center support
• Multi master mode - peer to peer communication
• High Availability
• Linear scalability
• Collection datatypes
• Faster updates
• Standard Monitoring Dashboard

Out of Scope

•      joins
•      group by
•      order by
•      secondary index - not performan

Important concepts

• seed-node
• Gossips
• peer to peer
• initial_token
• listen_address
• rpc_address
• Compaction
• tombstone
• Quorum
• consistency
• replication factor
• snitch

Need to learn:

Installation 

Connect to Cassandra (via cql)

create key space/table/index

How to add node

How to remove node

How to repair a cluster

How to backup c* cluster
How to setup monitoring  (OPS center)

How to set replication factor
How to set consistency Level (via Driver)
How to set authenticaion in connection