Extract one column from a CSV

Assuming your CSV looks like this

“ID”,”DeviceName”,”IPAddress”,”LastStatusChange”
“211″,”customer name here”,”172.16.10.19″,”2018-08-1309:51:28″
“339″,”fue bah”,”10.13.0.1″,”2018-09-0619:28:30″

We can use Notepadd++ (or other tool) and some regex to extract the IP address.

Find what: ^(.*?),(.*?),(.*?),(.*?)$
Replace with: $3

Be sure to use regex if you’re on Windows/Notepad++!  This works with four “column” documents, to adjust it for more or less be sure to mirror the number of commas and use one – (.*?) – for each column.

CentOS 6 with Python 2.7

https://danieleriksson.net/2017/02/08/how-to-install-latest-python-on-centos/

 

wget http://python.org/ftp/python/2.7.14/Python-2.7.14.tar.xz
tar xf Python-2.7.14.tar.xz
cdPython-2.7.14
./configure–prefix=/usr/local–enable-unicode=ucs4–enable-shared LDFLAGS=”-Wl,-rpath /usr/local/lib”
make&&makealtinstall


wget https://bootstrap.pypa.io/get-pip.py

# Then execute it using Python 2.7 and/or Python 3.6:
python2.7 get-pip.py

# With pip installed you can now do things like this:
pip2.7 install [packagename]
pip2.7 install –upgrade [packagename]
pip2.7 uninstall [packagename]

Clean Config Output

One of the best alias ideas I’ve ever seen!  This strips all the comments and such out of a config file so you can see, generally, what it looks like to the program reading it.

# alias
alias cleanconfig=’sed -e ‘\”s/#.*//’\” -e ‘\”s/[ ^I]*$//’\” -e ‘\”/^$/ d’\”’

# cleanconfig /etc/ntp.conf
driftfile /var/lib/ntp/drift
restrict default kod nomodify notrap nopeer noquery

RRDTool dump and restore

This issue typically comes up when moving data from a 32bit to a 64bit system.  The commands are very easy to work with for one file, but with many files and multiple directories (subdirectories) it can be cumbersome.  I found this post in a forum and found it extremely useful and powerful.

SRC: http://www.linuxquestions.org/questions/programming-9/script-to-dump-and-restore-rrd-files-608407/

This should be a foo.sh file and you should run it in /usr/share/app/data where the files are within data or below that.  It will convert and rename the file from rrd to xml.  Transfer the data over to the new system and run the same command (either no argument or look at the bottom for ‘help’).

#!/bin/sh

ECHO="echo -e"

rrdump ()
{
	for rrd in `find . -type f -name "*.rrd"`
		do
			xml=`echo $rrd | sed 's/.rrd//g'`
			rrdtool dump $rrd > $xml.xml
			rm $rrd
		done
}	

xmlimport ()
{
	for xml in `find . -type f -name "*.xml"`
        do
                rrd=`echo $xml | sed 's/.xml//g'`
                rrdtool restore $xml $rrd.rrd
                rm $xml
        done
}


case "$1" in
        dump)
                rrdump
		;;
        import)
                xmlimport
                ;;
        *)
                $ECHO "$0 takes one of two arguments\n"
		$ECHO "$0 dump -- dumps contents to xml\n"
		$ECHO "$0 import -- imports xml files to rrd\n"
                ;;
esac

Onboarding with SNMP via BASH

With the release of cnMaestro you can now ditch the CNS server *AFTER* you’ve upgraded your radios to 2.5+  The text files ap.txt and cpe.txt are simply one IP per line.  I got these from my network management software (a simple SQL query for all Cambium ePMP AP and Cambium ePMP CPE).  This was done in PC with:

SELECT e.DeviceType,
INET_NTOA(e.IPAddress + a.StartAddress) AS ‘IP Address’
FROM Equipment e
INNER JOIN Customer c ON c.CustomerID=e.EndUserID
LEFT OUTER JOIN AddressRange a ON e.IPType = a.AddressRangeID
WHERE e.Type LIKE ‘%Access Point%’
AND e.DeviceType LIKE ‘%Cambium ePMP AP%’
GROUP BY e.ID
ORDER BY c.CustomerID;

This simply requires SNMP access and snmputils to work.

#!/bin/bash

community=CHANGETHISCOMMUNITYHERE
server=https://cloud.cambiumnetworks.com
cambiumid=URCMPNYIDGOESHERE
onboardkey=048j03j03jf093jf0j309jf03jf

#use this to identify your radio versions (it needs 2.5+)
#while read -r radioip; do
# echo “Querying” $radioip
# snmpget -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.1.1.1.0
#done < “ap.txt”

#this is how you enable, set host/cambiumid/onboardkey, apply
while read -r radioip; do
echo “Querying” $radioip
snmpset -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.3.20.1.0 i 1
snmpset -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.3.20.2.0 s $server
snmpset -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.3.20.3.0 s $cambiumid
snmpset -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.3.20.4.0 s $onboardkey
snmpset -v2c -c$community $radioip .1.3.6.1.4.1.17713.21.4.4.0 i 1
done < “ap.txt” >> onboard.log

Migrating a Linux machine

In my case I needed to move a machine from a headless box to an ESXi server.  The VMWare tool does not support software raid.

I started by installing the OS on the new box with the same partition layout.  This takes care of all of the mbr and boot stuff.  The critical part, though, is / with the configuration, installed programs and any amount of data.  I used nc to do this and did it very painlessly.

#server you’re sending files to
nc -l 1024 > backup4.tgz

#client you’re backing up
tar -cvpz –one-file-system / | nc neon.imaginenetworksllc.com 1024

#boot the new box aka new client with a linux live cd and mount the partition
mkdir /mnt/sda3
mount /dev/sda3 /mnt/sda3
nc -l 1024 | tar -xvpzf – -C /mnt/sda3

#on that backup server
cat backup4.tgz | nc NEW.BOX.COM 1024

In my case I just installed the OS on the box so it had grub and the boot partition all done. What makes this box is the files in /.
Moving from ext3 partitions on md0 devices directly to sda1 I have to tell it to mount correctly.
*/etc/fstab
*eth0.bak has the ip config

*nload was nice to monitor network usage

*lots of help from https://help.ubuntu.com/community/BackupYourSystem/TAR

Visualizing disk usage in Linux – CLI/command line

http://gt5.sourceforge.net/

This project is simply amazing.  The download includes the binary and other files.  Simply execute gt5 from where it is extracted or more wisely place it in /bin/gt5 so you can run it from anywhere.  You can specify any directory (ie: gt5 /).

*stores previously ran information and will show you the different (in bytes)
*very nice easy interface
*largest (percentage) at the top to narrow down where the disk usage is
*runs on nearly any system with included utilities (du for example)

Output is done in HTML and easily shown after parsing with the included tool, again with the simple execution of gt5 /

It took a bit of digging to find a good program that would work directly in the shell for headless machines.  This one is simply fantastic.

 

EDIT:
gt5 is still pretty good but it’s not as easy/fast to get installed.  Recently I’ve been using ncdu thanks to this question/answer.  There are CentOS and Debian packages in their repos.  ncdu homepage.

Installing vSphere converter and other .NET applications

error 1935 0x80070bc9

If you see these you will need to reboot your machine.  It fixed it for me on my attempt to install Vmware vSphere Converter 6.0.0.  It appears there is a lock of some sort that prevents the installer to complete.  All other search came up dry for the solution so I thought I would post it here for some visibility.