Folder Sharing in Ubuntu Linux with Samba

A Samba file server enables file sharing across different operating systems over a network. It lets you access your desktop files from a laptop and share files with Windows.



Installation -
sudo apt update
sudo apt install samba


Check Installation -
whereis samba

The following should be its output:

samba: /usr/sbin/samba /usr/lib/samba /etc/samba /usr/share/samba /usr/share/man/man7/samba.7.gz /usr/share/man/man8/samba.8.gz


Create samba conf file backup
sudo cp /etc/samba/smb.conf /etc/samba/smb.conf.orig


Add Folder entry in /etc/samba/smb.conf which you want to share as below -

[Friends]                        # share folder name
    path = /home/atul/Desktop/Friends   # folder path
    read only = no                    
    browsable = yes


Start Samba Service
sudo systemctl restart smbd
sudo service smbd restart


Check if all working as configured
testparm


Now, Set password Samba User -
sudo smbpasswd -a <user_name>   # user_name should be existing system user


Share the Folder link to respective users -
Windows User - \\ip-address\Friends
Ubuntu User  - smb://ip-address/Friends

Like the below page to get update  
Facebook Page      Facebook Group      Twitter Feed      Google+ Feed      Telegram Group     

Read More

How to export the datastage job in individual dsx

This script will export the datastage jobs into individual dsx files as sometimes, we need individual dsx to maintain the code version or syncing it with SVN.

Manually export of each datastage job is quite hectic and boring hence this script :-)

This script requires little bit information from you and set you free for your other chores

Prerequisites for this script -

1. Job List to Export
2. DataStage Client Installation Location
3. DataStage Credential

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Crontab for Windows #iLoveScripting

While working on one of my project, I required to take backup of all the work which I have completed coz workplace is shared among many developers.
     So being a Linux person, I was looking for something simple like Crontab but ended with Windows Task Scheduler. 

Tool is simple but not as Linux Crontab But it did the work asked by me :-)

How to Use Task Scheduler - 

• Login with Admin privilege user account  
• Open Run and "Taskschd.msc"
•  Or  Go to Start --> Control Panel --> System and Maintenance --> Administrative Tools --> Task Scheduler
• 
• Click on "Create Task" on right hand side
• 
• This will open a Wizard to create Task
• Fill the Task Name, Owner, Privilege and Configured for as below - 
• 
• Now, Click on Next Tab - Trigger, Here you have to define the time when you want to execute the program
• 
• You can fill the different Setting to customize your schedule.
• Now, Click on "Action Tab"
• In this tab, you have to define the action, such as when triggered what program/script should be execute
• 
• Click OK
• You can see your task created under "Task Scheduler Library"
• 

For More details on Task Scheduler, You can visit below links -
https://technet.microsoft.com/en-us/library/cc748993(v=ws.11).aspx

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

How to get DataStage Project Settings

DSListProjectSettings.bat is used to report all available settings of existing projects.

USAGE: DSListProjectSettings <SERVER> <USER> <PASSWORD> <PROJECT>

The DSListProjectSettings batch file reports all available project settings and environment variable names and values to standard output. The first section shows available DataStage project properties and their settings. The second section shows environment variable names and values. When run successfully, each section will report a status code of zero.

..

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Import the jobs from DS windows client #iLoveScripting

As we have discussed a script which can export the datastage jobs from your client system (http://bit.ly/2frNPKj) likewise we can write another one to import the jobs. Let's see how -

DsImportJobsClient.bat :

This Script read all the *.dsx job name from the specified Directory and Sub-Directory and import to the Specified project. It can also build (Only BUILD) the existing package created on Information Server Manager and send it to the specified location on client machine.

To use the build feature you need to make sure the package has been created with all the needed jobs, saved and closed. Only update to the selected job will be taken care automatically. To add/delete a job, you need to do manually.

Modify the Import.properties and ImportJobList.txt file and Go the .bat dir and then execute the importAndBuild.bat.

Import.properties :


ImportJobList.txt :


DsImportJobsClient.bat :

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Export the jobs from DS windows client

Datastage jobs Export/Import are occasional activity (Deployment time :-)) for a developer But it becomes very tedious if the job list are long or it's daily routine to export or import jobs.

So I have written a batch script (windows script) which we can execute from the Client Machine (where datastage clients are installed) and automate this process.

DSjobExportClient.bat :
Export Script read the job name from the file (ExportJobList.txt) and exports the jobName.dsx from the project to the export base location and maintain the folder structure specified in the ExportJobList.txt file. File "ExportJobList.txt" and “Export.properties” should be updated before running the export script.

Copy the above file on any location of your DS windows client machine and update the "ExportJobList.txt" and “Export.properties” files.
-    Export.properties
-    ExportJobList.txt
-    ExportJobs.bat   



Export.properties :


ExportJobList.txt :


DsExportJobsClient.bat :

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Jenkins with Windows #1

One of my team member is assigned to install and configure "jenkins" on our server so out of curiosity, I asked what is this now ?? but didn't get a satisfactory answer :-) so I thought of having my hand dirty with it. Here, I am sharing whatever I learn.

What is Jenkins:-
Wikipedia sources says,  Jenkins is an open source automation server written in Java. Jenkins helps automating the non-human part of the whole software development process, with now common things like Continuous Integration, but by further empowering teams to implement the technical part of a Continuous Delivery.

What is Continuous Integration & Continuous Delivery:-
CI is a process that most developers follow to keep their code base intact. It's mostly a common practice when you work in a group environment. For example, an analogy for this would be constructing a new home. There will be multiple contractors working on the site. So, if we have installed the window glasses and the paint person comes in and paints the house there are high chances that he will drop some paint on the glasses or end up breaking the glass. So, the inspector comes and checks it every day to see if something broke. The same process is applied for constructing a new code. CI system gathers all your code from different developers and makes sure it compiles and build fine. This is good. But, not complete. I will get to that once I complete talking about Jenkins.

Jenkins is the inspector in the analogy. Jenkins is nothing but a middle man between your code repo and your build server. It checks for changes on your server every few minutes. If it found them, it gathers them and sends them to your build server. That's what Jenkins is.

Basically Continuous Integration is the practice of running your tests on a non-developer machine automatically everytime someone pushes new code into the source repository.

This has the tremendous advantage of always knowing if all tests work and getting fast feedback. The fast feedback is important so you always know right after you broke the build (introduced changes that made either the compile/build cycle or the tests fail) what you did that failed and how to revert it.

If you only run your tests occasionally the problem is that a lot of code changes may have happened since the last time and it is rather hard to figure out which change introduced the problem. When it is run automatically on every push then it is always pretty obvious what and who introduced the problem.

Built on top of Continuous Integration are Continuous Deployment/Delivery where after a successful test run your instantly and automatically release the latest version of your codebase. Makes deployment a non-issue and helps you speed up your development.


                              Jenkins offers the following major features out of the box, and many more can be added through plugins:

Developer time is focused on work that matters — Much of the work of frequent integrations is handled by automated build and testing systems, meaning developer time isn't wasted on large-scale error-ridden integrations.

Software quality is improved — Any issues are detected and resolved almost immediately, keeping software in a state where it can be safely released at any time.

Faster Development - Integration costs are reduced both because serious integration issues are less likely and because much of the work of integration is automated.

Easy installation: Just run java -jar jenkins.war, deploy it in a servlet container. No additional install, no database. Prefer an installer or native package? We have those as well.

Easy configuration: Jenkins can be configured entirely from its friendly web GUI with extensive on-the-fly error checks and inline help.

Rich plugin ecosystem: Jenkins integrates with virtually every SCM or build tool that exists.

Extensibility: Most parts of Jenkins can be extended and modified, and it's easy to create new Jenkins plugins. This allows you to customize Jenkins to your needs.

Distributed builds: Jenkins can distribute build/test loads to multiple computers with different operating systems. Building software for OS X, Linux, and Windows? No problem.


Check out the part 2 for Installation.



Sources:
https://jenkins.io/ https://en.wikipedia.org/wiki/Jenkins_(software) http://stackoverflow.com https://www.quora.com

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Windows & Linux - Command Comparison

I am a linux command line geek and when I am working on windows I miss it as there are lot of kung-fu which we can do easily in linux terminal than any GUI tool. So thought of creating a command equivalent list of Windows and Linux.

let me know if there is any daily command which I missed to add here. Hoping this will help you guys.

Command's Purpose	MS-DOS	Linux	Basic Linux Example
Copies files	copy	cp	cp thisfile.txt /home/thisdirectory
Moves files	move	mv	mv thisfile.txt /home/thisdirectory
Lists files	dir	ls	ls
Clears screen	cls	clear	clear
Closes shell prompt	exit	exit	exit
Displays or sets date	date	date	date
Deletes files	del	rm	rm thisfile.txt
"Echoes" output to the screen	echo	echo	echo this message
Edits text files	edit	gedit	gedit thisfile.txt
Compares the contents of files	fc	diff	diff file1 file2
Finds a string of text in a file	find	grep	grep word or phrase thisfile.txt
Displays command help	command /?	man or info	man command
Creates a directory	mkdir, md	mkdir	mkdir directory
Deletes an existing directory	rmdir, rd	rmdir	rmdirt directory
Views contents of a file	more	less	less thisfile.txt
Renames a file	ren	mv	mv thisfile.txt thatfile.txt
Displays your location in the file system	chdir	pwd	pwd
Changes directories with a specified path (absolute path)	cd pathname	cd pathname	cd /directory/directory
Changes directories with a relative path	cd..	cd ..	cd ..
Displays the time	time	date	date
Shows amount of RAM in use	mem	free	free
Displays the contents of a file	type	cat	type filename
Reverse IP lookup	nslookup	nslookup
Checking the network status	netstat	netstat	netstat
Pinging any system	ping	ping	ping 127.0.0.1
Running Process	tasklist	ps	ps -ef
Get login username	net session	who	who
Tracing IP	tracert	traceroute
Sets file permissions	Attrib	chmod	chmod perm1 perm2
Display OS version	ver	uname -a	uname -a
Sorts lines in a file	sort	sort	sort filename
Compare two files	diff	fc	fc file1 file2
OS scheduler	at	crontab	crontab -l
FTP	ftp	ftp	ftp user@host

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

jupyter notebook tip #1

How we can start jupyter notebook from a specific directory?

Below, I am sharing a very tiny batch (and linux shell) script which will kick off the jupyter notebook from a particular directory with just a double click...

I hope, this will give you a little ease while working on IPython :-)

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Python - Get line no with print statement

Sometime when we are working on some code and there are lot of print statement, It's difficult to check for which print statement produce this statement.

It helps a lot while debugging the code and helped me to create better python learning script which will generate the output with the code line no which is easy to relate.

Code:

from inspect import currentframe

def lno():
    cf = currentframe()
    val = str(cf.f_back.f_lineno)+". "
    return val

print "this is Me", lno()
print lno(), "Hi! there"

Output:

this is Me  8.

9.  Hi! there

** This is a little overhead on the code as with each print statement the function is being called. So use it wisely :-)

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://datagenx.slack.com/messages/datascience/

Read More

Where to get DataStage Scheduled jobs ?

Did you guys ever try to find out how the scheduler works when we schedule jobs by DataStage scheduler? Interesting na !!

So, let's have a discussion how this is happening.

DataStage itself doesn't contain any scheduling system or application but it is using OS-level scheduling techniques which is OS-level job scheduler.

Unix/Linux-
If the DataStage server is on Linux/Unix/*nix OS. That's "at" and "cron"

'at -lv' or 'at -l' will show any jobs that are scheduled to run once, 'crontab -l' will show jobs that are scheduled to repeat


Windows-
On Windows, the jobs become entries in the Scheduled Tasks control panel. In GUI, you can't understand much. But we can extract and dump this information to see it better.

'schtasks /query /v /fo list' command give you a proper formatted information about the scheduled jobs.

Windows has nice output format options, and (with the verbose switch) lots of info, including the next run time, last run time and last result, so you can tell if a job actually ran when it was supposed to.  You may need to use a batch script to filter out the specific entry.

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More

Python, IPython, Jupyter notebook, Graphlab Installation on Windows

In "Python Installation from Source in Linux" and "Data Science Tools Installation in Linux" we have seen, how to install these tools on linux, Today we will learn how to setup these tools on Windows -

Python Installation:

1. Download the Python Windows installer from here -> https://www.python.org/downloads/release/python-2711/

2. Install it as we install any software on windows

3. Now, setup the Environment Variable -
a.              If you haven’t played with environment variables before, just stick to following these instructions as you can set them up through the Windows GUI.
b.             Right click on "My Computer", select "Properties" > "Advanced system settings" and click on the "Environment Variables" button
c.             In the System Variables box, find the variable called "path" and click on the "Edit…" button
d.             In the "Variable value" box, at the end of the entry, add the following text: ;C:\Python27;C:\Python27\Scripts (change the path as per your installation)
e.             Click "OK" a couple of times and hey presto, your environment variables are set up.
f.              Open cmd and type command 'python', if you get the python prompt we are good else check the steps once again.

4. The next step in the process is to set up easy_install and so we need to go to the setuptools page (links to version 0.8) and download the ez_setup.py script. You can download it from here -> https://bitbucket.org/pypa/setuptools/raw/0.8/ez_setup.py. and put this in python script directory (C:\Python27\Scripts)

5. Open a command prompt and type python ez_setup.py install – you’ll see a load of code whizz by which will hopefully end as follows;

C:\Python27> python ez_setup.py install
Processing dependencies for setuptools==0.8
Finished processing dependencies for setuptools==0.8
C:\Python27>

6. easy_install has now been set up and you can test to see if it is there, by typing easy_install in to a command prompt, which will throw an error about no URLs, you know that the tool has been set up successfully.

To use easy_install to get new libraries, just use the following syntax: easy_install <library name>


IPython Installation:

C:\Python27> easy_install ipython

Jupyter notebook Installation

C:\Python27\Scripts> pip install jupyter

You can run the jupyter notebook as below -

C:\Python27\Scripts>jupyter notebook

Graphlab Create Installation

C:\Python27\Scripts> pip install --upgrade --no-cache-dir https://pypi.python.org/packages/source/G/GraphLab-Create/GraphLab-Create-1.7.1.tar.gz#md5=caa4b1f78625a278dd016400d15bc5bd

Like the below page to get update  
https://www.facebook.com/datastage4you
https://twitter.com/datagenx
https://plus.google.com/+AtulSingh0/posts
https://groups.google.com/forum/#!forum/datagenx

Read More