Friday, October 23, 2015

Linux - Schedule Tasks Using Crontab

Schedule Tasks on Linux Using Crontab

If you've got a website that's heavy on your web server, you might want to run some processes like generating thumbnails or enriching data in the background. This way it can not interfere with the user interface. Linux has a great program for this called cron. It allows tasks to be automatically run in the background at regular intervals. You could also use it to automatically create backups, synchronize files, schedule updates, and much more. Welcome to the wonderful world of crontab.

 

Crontab

The crontab (cron derives from chronos, Greek for time; tab stands fortable) command, found in Unix and Unix-like operating systems, is used to schedule commands to be executed periodically. To see what crontabs are currently running on your system, you can open a terminal and run:

$ sudo crontab -l

To edit the list of cronjobs you can run:

$ sudo crontab -e

This wil open a the default editor (could be vi or pico, if you want you canchange the default editor) to let us manipulate the crontab. If you save and exit the editor, all your cronjobs are saved into crontab. Cronjobs are written in the following format:

* * * * * /bin/execute/this/script.sh

 

Scheduling explained

As you can see there are 5 stars. The stars represent different date parts in the following order:

  • minute (from 0 to 59)
  • hour (from 0 to 23)
  • day of month (from 1 to 31)
  • month (from 1 to 12)
  • day of week (from 0 to 6) (0=Sunday)

 

Execute every minute

If you leave the star, or asterisk, it means every. Maybe that's a bit unclear. Let's use the the previous example again:

* * * * * /bin/execute/this/script.sh

They are all still asterisks! So this means execute /bin/execute/this/script.sh:

  • every minute
  • of every hour
  • of every day of the month
  • of every month
  • and every day in the week.

In short: This script is being executed every minute. Without exception.

 

Execute every Friday 1AM

So if we want to schedule the script to run at 1AM every Friday, we would need the following cronjob:

0 1 * * 5 /bin/execute/this/script.sh

Get it? The script is now being executed when the system clock hits:

  • minute: 0
  • of hour: 1
  • of day of month: * (every day of month)
  • of month: * (every month)
  • and weekday: 5 (=Friday)

 

Execute on workdays 1AM

So if we want to schedule the script to Monday till Friday at 1 AM, we would need the following cronjob:

0 1 * * 1-5 /bin/execute/this/script.sh

Get it? The script is now being executed when the system clock hits:

  • minute: 0
  • of hour: 1
  • of day of month: * (every day of month)
  • of month: * (every month)
  • and weekday: 1-5 (=Monday til Friday)

 

Execute 10 past after every hour on the 1st of every month

Here's another one, just for practicing

10 * 1 * * /bin/execute/this/script.sh

Fair enough, it takes some getting used to, but it offers great flexibility.

 

Neat scheduling tricks

What if you'd want to run something every 10 minutes? Well you could do this:

0,10,20,30,40,50 * * * * /bin/execute/this/script.sh

But crontab allows you to do this as well:

*/10 * * * * /bin/execute/this/script.sh

Which will do exactly the same. Can you do the the math? ; )

 

Special words

For the first (minute) field, you can also put in a keyword instead of a number:

  • @reboot Run once, at startup
  • @yearly Run once a year "0 0 1 1 *"
  • @annually (same as @yearly)
  • @monthly Run once a month "0 0 1 * *"
  • @weekly Run once a week "0 0 * * 0"
  • @daily Run once a day "0 0 * * *"
  • @midnight (same as @daily)
  • @hourly Run once an hour "0 * * * *"

Leaving the rest of the fields empty, this would be valid:

@daily /bin/execute/this/script.sh

 

Storing the crontab output

By default cron saves the output of /bin/execute/this/script.sh in the user's mailbox (root in this case). But it's prettier if the output is saved in a separate logfile. Here's how:

*/10 * * * * /bin/execute/this/script.sh >> /var/log/script_output.log 2>&1

 

Explained

Linux can report on different levels. There's standard output (STDOUT) and standard errors (STDERR). STDOUT is marked 1, STDERR is marked 2. So the following statement tells Linux to store STDERR in STDOUT as well, creating one datastream for messages & errors:

2>&1

Now that we have 1 output stream, we can pour it into a file. Where >will overwrite the file, >> will append to the file. In this case we'd like to to append:

>> /var/log/script_output.log

 

Mailing the crontab output

By default cron saves the output in the user's mailbox (root in this case) on the local system. But you can also configure crontab to forward all output to a real email address by starting your crontab with the following line:

MAILTO="yourname@yourdomain.com"

 

Mailing the crontab output of just one cronjob

If you'd rather receive only one cronjob's output in your mail, make sure this package is installed:

$ aptitude install mailx

And change the cronjob like this:

*/10 * * * * /bin/execute/this/script.sh 2>&1 | mail -s "Cronjob ouput" yourname@yourdomain.com

 

Trashing the crontab output

Now that's easy:

*/10 * * * * /bin/execute/this/script.sh > /dev/null 2>&1

Just pipe all the output to the null device, also known as the black hole. On Unix-like operating systems, /dev/null is a special file that discards all data written to it.

 

Caveats

Many scripts are tested in a BASH environment with the PATH variable set. This way it's possible your scripts work in your shell, but when run from cron (where the PATH variable is different), the script cannot find referenced executables, and fails.

It's not the job of the script to set PATH, it's the responsibility of the caller, so it can help to echo $PATH, and put PATH=<the result> at the top of your cron files (right below MAILTO).

Taken From: http://kvz.io/blog/2007/07/29/schedule-tasks-on-linux-using-crontab/

Sunday, October 18, 2015

Cisco - Dual WAN Internet with Failover (NAT & Routes)

In the past I had the need to implement failover between two connections to the Internet on the same router, at the time I implemented it using EEM Scripts which wasn’t the most elegant solution, so here I’m going to show you a  a better solution to do this.

Implementing the failover mechanism at first glance seems easier with one router than with two, but that is not the case, with two routers you can have them configured normally with the adition of VRRP/HSRP to do the failover between the routers.

With only one router you are going to have two aditional problems:

  • Changing the route from the primary to the secondary Internet access
  • Changing the NAT overload to the Interface towards the Secondary ISP / WAN (this was the part that I implemented with EEM scripts)

the first you can easily solve with a floating static route (secondary route) and a track / ip sla (to remove the primary route when the connectivity to the primary ISP is lost).

The second one is harder, you can have two NAT rules with two interfaces towards the two ISPs:

ip nat inside source 130 interface FastEthernet0/0 overload
ip nat inside source 131 interface FastEthernet1/0 overload

but selecting the one as the active one is the tricky part.

Even if the interface towards the primary ISP were to fail and become shutdown, the NAT rule remains active.

The cenario bellow has two different ISPs for for the WAN accesses, but its the sames as having two different connectivities via the same ISP (eg. Primary via Fiber Optic | Secondary via 4G)

Cenario

Logical View

TOP17

Fisical View

TOP18

You can download the lab fully implemented here:

it was implemented on GNS3 v1.2.1.

 

Configuration

 

PC1
====================================================

enable
conf t

hostname PC1

interface FastEthernet0/0
description *** Link to CPE1 ***
ip address 192.168.1.1 255.255.255.0
no shutdown

ip route 0.0.0.0 0.0.0.0 192.168.1.254 name DefaultGW
 
do write
 
 
CPE1
====================================================

enable
conf t

hostname CPE1

interface FastEthernet0/0
description *** Link to ISP1 ***
ip address 11.0.0.2 255.255.255.252
ip nat outside
no shutdown
 
interface FastEthernet0/1
description *** Link to ISP2 ***
ip address 22.0.0.2 255.255.255.252
ip nat outside
no shutdown
 
interface FastEthernet1/0
no switchport
description *** Link to PC1 ***
ip address 192.168.1.254 255.255.255.0
ip nat inside
no shutdown

 
!-- Select the Route - via ISP1 or ISP2 -------------------
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! The first route makes all the traffic flow via
! ISP1, but this is conditioned to track 10, that
! detects the  connectivity to ISP1.
! If track 10 fails the route is removed from the
! routing table.
!
!
! The second route has an higher administrative
! distance (worst), and as long as the first rule
! is available this rule is never inserted on
! the routing table (aka floating static route)
!
! If the first route disapears because the track
! failed then the second route is inserted in the
! routing table, and all traffic will flow via ISP2
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

ip route 0.0.0.0 0.0.0.0 11.0.0.1 track 10 name Default-Primary
ip route 0.0.0.0 0.0.0.0 22.0.0.1 250 name Default-Secondary

ip sla 10
icmp-echo 11.0.0.1 source-interface FastEthernet0/0
frequency 5
ip sla schedule 10 life forever start-time now

track 10 ip sla 10 reachability
!show track brief
!show track 10


!-- Change the NAT Interface to Reflect the Active Route --
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Changing the routes isn't enough, we also
! need to change the NAT rule, because each route
! implies a different exit interface.
!
! To select which NAT rule will be used for each
! route, we used route maps instead of an ACL
! to identify traffic (active the rule).
!
! These route maps match the LAN traffic, plus
! the current next hop to forward the traffic thus
! selecting the correct NAT rule for the current
! active route.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

ip nat inside source route-map RM-ISP1-PRIMARY interface FastEthernet0/0 overload
ip nat inside source route-map RM-ISP2-PRIMARY interface FastEthernet0/1 overload


access-list 130 remark *** Traffic for The Internet (NAT) ***
access-list 130 permit ip 192.168.1.0 0.0.0.255 any

route-map RM-ISP1-PRIMARY permit 10
match ip address 130
match interface FastEthernet0/0        !--> Match the exit interface of the route

route-map RM-ISP2-PRIMARY permit 10
match ip address 130
match interface FastEthernet0/1        !--> Match the exit interface of the route
!show route-map


!-- Simulate a Failure Along The Way ----------------------
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! To simulate the failure to see the topology
! changing to the secondary access, we are going
! to use a route to force the track/ip sla to fail.
!
! This route will force all the connectivity test
! traffic destined for ISP1(11.0.0.1) to go to NULL
! which is a black hole. Like this ISP1 will never
! get the icmp echos requests from the ip sla test
! or respond to it, thus simulating a connectivity
! failure towards ISP1
!
! NOTE: It takes a couple of seconds to change
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!-- Failure Sim + Debug/Tshoot --
do debug ip nat
ping 77.0.0.1
show ip route
!S*    0.0.0.0/0 [1/0] via 11.0.0.1

ip route 11.0.0.1 255.255.255.255 NULL0 name FailureSim

ping 77.0.0.1
show ip route
!S*    0.0.0.0/0 [250/0] via 22.0.0.1

! Both Pings will work but notice that the default
! route is diferent (diferent next hop)


!-- Restore to Normal Operation --
no ip route 11.0.0.1 255.255.255.255 NULL0 name FailureSim
do no debug all


ISP1-PRIMARY
====================================================

enable
conf t

hostname ISP1-PRIMARY

interface FastEthernet0/0
description *** Link to CPE1 ***
ip address 11.0.0.1 255.255.255.252
no shutdown

interface FastEthernet0/1
description *** Link to CPE2 ***
ip address 11.0.0.5 255.255.255.252
no shutdown
 
 
interface FastEthernet1/0
no switchport
description *** Link to ISP2 ***
ip address 22.0.0.6 255.255.255.252
no shutdown

    
ip route 77.0.0.0 255.255.255.248 11.0.0.6 name PublicIPs
ip route 22.0.0.0 255.255.255.252 22.0.0.5 name NatedLAN-viaIPS2

do write


ISP2-SECONDARY
====================================================

enable
conf t

hostname ISP2-SECONDARY

interface FastEthernet0/1
description *** Link to CPE1 ***
ip address 22.0.0.1 255.255.255.252
no shutdown
 
interface FastEthernet0/0
description *** Link to ISP1 ***
ip address 22.0.0.5 255.255.255.252
no shutdown

ip route 0.0.0.0 0.0.0.0 22.0.0.6 name Default
 
do write
 
 
CPE2
====================================================

enable
conf t

hostname CPE2

interface FastEthernet0/1
description *** Link to ISP1 ***
ip address 11.0.0.6 255.255.255.248
no shutdown
 
interface FastEthernet1/0
description *** Link to Internet Server ***
no switchport
ip address 77.0.0.6 255.255.255.248

ip route 0.0.0.0 0.0.0.0 11.0.0.5 name Default

do write


INTERNET SERVER
====================================================

enable
conf t

hostname INTERNET-SERVER

interface FastEthernet0/0
description *** Link to CPE1 ***
ip address 77.0.0.1 255.255.255.248
no shutdown
 
ip route 0.0.0.0 0.0.0.0 77.0.0.6 name Default

do write



Related Links

Sunday, October 11, 2015

Linux/Raspberry - Send Emails (SMTP Setup - Gmail)

 

SMTP Mail Setup

Many times you want to have the ability to send email from processes on your Raspberry Pi to email addresses out on the network. Adding email to your Raspberry Pi is pretty simple. You can use the following three packages for some simple mail capabilities.

It is assumed that you have networking working already...

 

Loading the packages

sudo apt-get install ssmtp
sudo apt-get install mailutils
sudo apt-get install mpack

Setting up the defaults for SSMTP

sudo nano /etc/ssmtp/ssmtp.conf

Now edit the fields:

AuthUser=youruserid@gmail.com
AuthPass=userpass
FromLineOverride=YES
mailhub=smtp.gmail.com:587
UseSTARTTLS=YES

Using Email Now

echo "sample text" | mail -s "Subject" username@domain.tld

Sending Attachments

mpack -s "test" /home/pi/test/somefile.ext username@domain.tld

Taken From:

Friday, October 9, 2015

How to Mount CD/DVDs Images - Windows, Mac, and Linux

Disc images have become more useful than ever on modern PCs that often lack CD and DVD drives. Create ISO files and other types of disc images and you can “mount” them, accessing the virtual discs as if they were physical discs inserted into your computer.

You can also use these image files to burn copies of the original discs later, creating duplicate copies. Disc image files contain a complete representation of a disc.

 

Windows

Windows 10 allows you to mount both .ISO and .IMG disc image files without any third-party software. Just double-click a .ISO or .IMG disc image you want to make available. If this doesn’t work, you should be able to click the “Disk Image Tools” tab on the ribbon and click “Mount.” It will appear under Computer as if it were inserted into a physical disc drive.

This feature was added back in Windows 8, so it will also work on Windows 8 and 8.1.

To unmount the disc later, right-click the virtual disc drive and select “Eject.” The disc will be unmounted and the virtual disc drive will disappear from the Computer window until you mount a disc in it again.

clip_image001

To mount ISO or IMG images on Windows 7 — or to mount images in other formats, such as BIN/CUE, NRG, MDS/MDF, or CCD — we recommend the free, open-source, and simple WinCDEmu utility.

Just right-click an image file after installing it, click “Select drive letter & mount,” and you can mount other types of images Windows doesn’t support.

Some other third-party utilities have additional support for emulating various copy-protection technologies, allowing copy-protected discs to function normally. However, such techniques are being phased out and aren’t even supported by modern versions of Windows.

clip_image002

 

Mac

On a Mac, double-clicking common disc image formats will mount them. This is why you can simply double-click a downloaded .DMG file to access its contents and install Mac applications, for example.

The DiskImageMounter application that handles this can also mount .ISO, .IMG, .CDR, and other types of image files. Just double-click the file to mount it. If this doesn’t work, Option-click or right-click a file, point to “Open With,” and select “DiskImageMounter.

When you’re done, just click the “Eject” button next to the mounted image in the Finder’s sidebar to eject it and unmount it — just like you’d unmount a .DMG image when you’re done with it.

clip_image003

You can also try mounting the disc image file by opening the Disk Utility application. Press Command+Space, type Disk Utility, and press Enter to open it. Click the “File” menu, select “Open Image,” and select the disc image you want to mount.

clip_image004

 

Linux

Ubuntu’s Unity desktop and GNOME include an “Archive Mounter” application that can mount ISO files and similar image files graphically. To use it, right-click an .ISO file or another type of disc image, point to Open With, and select “Disk Image Mounter.”

You can later unmount the image by clicking the eject icon next to the mounted image in the sidebar.

clip_image005

You can also mount an .ISO file or another disc image with a Linux terminal command. This is particularly useful if you’re just using the command line, or if you’re using a Linux desktop that doesn’t provide a tool to make this easy. (Of course, graphical tools for mounting ISO files and similar images may be available in your Linux distribution’s software repositories.)

To mount an ISO or IMG file on Linux, first open a Terminal window from your Linux desktop’s applications menu. First, type the following command to create the /mnt/image folder. You can create practically any folder you like — you just have to create a directory where you’ll mount the image. The contents of the disc image will be accessible at this location later.

sudo mkdir /mnt/image

Next, mount the image with the following command. Replace “/home/NAME/Downloads/image.iso” with the path to the ISO, IMG, or other type of disc image you want to mount.

sudo mount -o loop /home/NAME/Downloads/image.iso /mnt/image

To unmount the disc image later, just use the umount command:

sudo umount /mnt/image

clip_image006

Some guides recommend you add “-t iso9660” to the command. However, this isn’t actually helpful - it’s best to let the mount command automatically detect the required file system.

If you’re trying to mount a more obscure type of disc image format that the mount command can’t automatically detect and mount in this way, you may need commands or tools designed specifically for working with that type of image file format.

This should “just work” on most modern operating systems, allowing you to mount and use ISO images and other common types of image files in a few clicks. Windows 7 users will have the toughest time, as it isn’t integrated into that older version of Windows, but WinCDEmu is a lightweight and easy way to accomplish this.

Taken From: