How to Run Batch Operation on Cluster Computer

If you are pursuing your higher studies in the field of machine learning, its very likely that you have to “deal” with big data. By “deal”, I mean you might have to extract features, or say for example, find the eigenvectors and eigenvalues of large matrices. Carrying out these types of simulations on real or synthetic data on typical desktop/laptops may take days to run. Its suggested that you run these simulations on your University’s super computer or cluster computer. I am assuming that you have access to such a cluster.

These clusters are typically multi-core computers with huge amount of RAMs and tremendous processing powers. But you cannot just run any code on these clusters. You need a batch script (SLURM) to utilize the resources. You have access to a front end node and through that node you have to submit your batch job. I will show how to run a MATLAB code on the cluster using the sbatch command. I am assuming that you are on MacOSX/Linux and can use the terminal. If you are on Windows, install PuTTY to emulate the terminal.

Continue reading

How to Get Working Internet on Linux Mint/Ubuntu from Behind A Squid Proxy

Almost all of us have to encounter proxies in our work environments. Sometimes those are useful (providing security and firewall), and sometimes those are just a pain. For example, at my work place, I have to go through a Squid proxy. In this tutorial, I will show how to set a Linux Mint/Ubuntu (and possibly other distros) computer to get “internet connection” from behind a proxy. By “internet connection”, what I mean is to get the Synaptic package manager (or APT) and other applications (like the browsers) send http requests through the proxy. Let’s get to it.

Continue reading

Home Networking: File Transfer

Many of us have two or more computers at our home and like to share files between these PCs. If you don’t have a huge PenDrive or portable HDD or a DVD writer, then you might have problem to transfer large files from one PC to another. You can easily create a Local Area Network (LAN) among the computers for sharing files among the PCs.

This article shows how to connect two PCs using a cross-over LAN cable for file sharing purposes.

Here are the descriptions of the PCs

PC1: Ubuntu 8.04 with a Tenda 10/100Mbps LAN card
PC2: Windows XP Profesional with Built-in 10/100Mbps LAN card

Continue reading