Friday, 2 February 2018

Install graphite on CentOS 7


1) Install and enable EPEL
sudo wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum install epel-release-latest-7.noarch.rpm

Tip:
To list all available packages under a repo called epel, enter:
$ sudo yum --disablerepo="*" --enablerepo="epel" list available

2) Install graphite related packages
sudo yum install -y graphite-web python-carbon

3) Modify storage schema file
sudo nano /etc/carbon/storage-schemas.conf

[default]
pattern= .*
retentions = 12s:4h, 2m:3d, 5m:8d, 13m:32d, 1h:1y

Now, restart carbon:
sudo service carbon-cache restart

OR
sudo systemctl enable carbon-cache
sudo systemctl start carbon-cache

4) Modify some settings in django graphite web application file:
sudo nano /etc/graphite-web/local_settings.py

# line 13: uncomment and specify any secret-key you like
SECRET_KEY = 'my_secret_key'

# line 23: uncomment and change to your timezone

TIME_ZONE = 'Asia/Calcutta'

5) Now, create django superuser and create database.
/bin/graphite-manage syncdb

Enter required user/password for superuser etc in the interactive session.

6) Configure apache for graphite:
Remove the default index page from apache:

  echo > /etc/httpd/conf.d/welcome.conf

Next, edit /etc/httpd/conf.d/graphite-web.conf and replace everything in the 'Directory "/usr/share/graphite/"' block with:

    Require all granted
    Order allow,deny
    Allow from all


Check if proper permissions to the Graphite directory:

sudo chown apache:apache /var/lib/graphite-web/graphite.db

permission should be: -rw-r--r-- and owner:apache:apache

And work around a bug related to building indexs with:

  touch /var/lib/graphite-web/index

permission should be: -rw-r--r-- and owner:root:root


Start Apache and enable auto-start:

sudo systemctl start httpd 
sudo systemctl enable httpd 


7) If firewall is enabled, allow port 80 access.

sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --reload

Useful links:
https://www.vultr.com/docs/how-to-install-and-configure-graphite-on-centos-7

Sunday, 24 December 2017

Process dump the attacker using logstash

Use linux "auditd" to monitor /etc/passwd file and generate a key "password_access".

$ sudo apt install auditd
$ auditctl -w /etc/passwd -p wa -k passwd_access

Force logstash core dump any process that causes auditd to write the "password_access" key.

Install gdb (gcore):
sudo apt install gdb

Modify the output section of your /etc/logstash/conf.d/00-output.conf:

output {
  if [key] == "password_access" {
      stdout {codec => json}
      exec { command => "gcore -o /tmp/dump-%{@timestamp} %{pid}"}
  }
}


Tuesday, 19 December 2017

Why you should not run multiple anti-virus products

I liked the advice from Emsisoft Anti-virus about why you should not run multiple anti-virus products.

* Potential conflicts/incompatibility issues

Modern anti-virus/anti-malware software acts like an extra protection layer that sits between the base of the operating system and the apps/programs that run on it. Developing this type of software is always challenging and requires many years of experience.Protection programs are created in different ways and it may cause unexpected crashes or freezes that are very difficult to resolve.

* Who will quarantine first

Since anti-virus products have real-time scanning enabled, it's a race between multiple anti-virus programs about quarantining potential malicious download/file scan and this may give unexpected results/errors.

* High resource usage

Since number of viruses/malwares are growing exponentially, size and complexity of anti-virus programs is also correspondingly growing and these now consume lot of CPU/storage resources. The situation will be further compounded if you use multiple anti-virus products.

So, in nutshell, avoid installing multiple anti-virus/anti-malware products as it’s not worth it. If you are happy with existing anti-virus software, stick with it.If you are unhappy with it, un-install it and then install a new one.

Ref -
* Don't run multiple anti-virus products - https://blog.emsisoft.com/2017/12/18/do-not-run-multiple-antivirus
* Latest independent tests of anti-virus products - https://www.av-comparatives.org/

Sunday, 10 December 2017

Fix "this webpage has a redirect loop" error in browser

On a daily basis, Internet as a whole suffers from many problems and one such problem that is sometimes encountered is

"The webpage has a redirect loop"

The other times, the redirect error manifests itself with the following details:

Error 310 (net::ERR_TOO_MANY_REDIRECTS)

As a result, every time you visit Gmail page in Firefox/Chrome, the redirection loop will prevent access to it.

When Chrome or Firefox starts complaining about redirect loops just do three things:

1) Check the clock and set appropriate Date/Time as per your timezone
2) Clear the Browser cache
3) Kill the browser settings and reset it to default state

By the way, the no of redirect vary depending on browser and are listed below:

  • Chrome 64bit version: 49, ↷ Version 62.0.3202.52 (Official Build) beta, 21 redirects
  • Chrome Canary 64bit, version: 49 ↷ 63.0.3239.6 (Official Build) canary, 21 redirects
  • Firefox 32-bit version: 43 ↷ 56.0, 20 redirects
  • Firefox 64-bit version: 43 ↷ 56.0, 20 redirects
  • IE version: 8 11 redirects via webpagetest.org
  • IE version: 9 121 redirects via webpagetest.org
  • IE version: 10 121 redirects via webpagetest.org
  • IE version: 11.0.9600.18792 110 redirects
  • Opera version: 28, ↷ 48.0.2685.35 (PGO) 21 redirects
  • Safari version: 5.1.7, 16 redirects
  • Google Nexus 5, Samsung Galaxy S4..S8, Galaxy Tab 4, 21 redirects
The latest firefox version - Quantum supports up to 40 redirects.

Ref -
https://stackoverflow.com/questions/9384474/in-chrome-how-many-redirects-are-too-many

Sunday, 19 November 2017

Save bash command history to syslog


# Increase history size
export HISTSIZE=5000

# In the commands given below - every time a new prompt is issued , bash history is appended to the file, then it is cleared from the current shell's memory,  and current shell reloads the history from the file.

$ export PROMPT_COMMAND="history -a; history -c; history -r; ${PROMPT_COMMAND}"

Another option is to export bash commands to syslog where the bash logs can be centralized and analyzed on demand.

Add the following snipplet to bashrc.

[root@psj]# vim /etc/bashrc

PROMPT_COMMAND=$(history -a)
typeset -r PROMPT_COMMAND

function log2syslog
{
   declare command
   command=$BASH_COMMAND
   logger -p local1.notice -t bash -i -- "$USER : $PWD : $command"

}
trap log2syslog DEBUG



Friday, 10 November 2017

Download gz file using python request module

Here is the quick script I wrote to download a gz file using python requests module:

#!/usr/bin/env python
import requests
import gzip
import logging
import sys
import StringIO
import zlib

# setup logging
logging.basicConfig(stream = sys.stdout, level = logging.ERROR)
log = logging.getLogger('threat-feeds-logger')

#proxy configuration
proxy_host='10.1.1.11'
proxy_port=3128
proxy_user = 'xxxx'
proxy_password = 'xxxx'

feed_url = 'https://foo.org/foo.gz'
proxy_dict = {
                'http':'http://%s:%s@%s:%s' % (proxy_user, proxy_password, proxy_host, proxy_port),
                'https':'http://%s:%s@%s:%s' % (proxy_user, proxy_password, proxy_host, proxy_port)
            }
try:
    response = requests.get(feed_url,proxies = proxy_dict)
except Exception as e:
    log.error("Error while getting data from url - %s" %feed_url)

if response.status_code == 200:
    buf_data = StringIO.StringIO(response.content)
    f = gzip.GzipFile(fileobj=buf_data)
    for row in f.readlines():
       print row
 

Thursday, 9 November 2017

How to find google IP address range

I wanted to track google IP range in one of the investigation. So, I did this:

[admin@psj ~]$ nslookup -type=txt _spf.google.com 8.8.8.8
Server:        8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
_spf.google.com    text = "v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

Authoritative answers can be found from:
google.com    nameserver = ns2.google.com.
google.com    nameserver = ns3.google.com.
google.com    nameserver = ns1.google.com.
google.com    nameserver = ns4.google.com.
ns2.google.com    internet address = 216.239.34.10
ns3.google.com    internet address = 216.239.36.10
ns1.google.com    internet address = 216.239.32.10
ns4.google.com    internet address = 216.239.38.10

Run a nslookup for each one:
$ nslookup -q=TXT _netblocks.google.com 8.8.8.8
$ nslookup -q=TXT _netblocks2.google.com 8.8.8.8


[admin@psj ~]$ nslookup -type=txt _netblocks.google.com
Server:        8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
_netblocks.google.com    text = "v=spf1 ip4:64.18.0.0/20 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:207.126.144.0/20 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all"

Authoritative answers can be found from:
google.com    nameserver = ns2.google.com.
google.com    nameserver = ns3.google.com.
google.com    nameserver = ns4.google.com.
google.com    nameserver = ns1.google.com.
ns2.google.com    internet address = 216.239.34.10
ns3.google.com    internet address = 216.239.36.10
ns1.google.com    internet address = 216.239.32.10
ns4.google.com    internet address = 216.239.38.10

You can also use whois query to find out network blocks assigned to google.
$ whois 74.125.127.108