Tag Archives: Cloud

Essentials of working with Python cloud (Ubuntu)

In most of the data science applications, it comes very handy to be able to run code on the cloud. Be it a simple demonstration of a functionality that we want to make accessible for a potential client or an end-to-end implementation of let’s say a predictive model, the accessibility of cloud-based solutions is a definitive asset. However, running code on the cloud does have its pitfalls, which can discourage many from taking advantage of it.

This is why I have decided to share our experience with working on the cloud. In this post, I will specifically give a summary of functionalities that can help to run a python script on the Ubuntu cloud.python_on_cloud

Running a python script on the cloud, can become much more bothersome than the development on our local computer, especially if we are using a standard SSH connection. Fortunately, to make our lives easier, there are a couple of functionalities that we can use.

1. argparse (python) – to run the script with various input arguments
2. tmux (unix) – to run sessions without the need to have a permanent SSH connection
3. cron (unix) – to run the scripts with a predefined frequency
4. SimpleHTTPs (python) – lightweight webserver for providing access to files to users that don’t have access to our cloud

Continue reading

Advertisements

Data Scientists toolbox: Cloud set-up using Ansible

Given the size of data and complexity of processing, many Data Science projects require scalability that can be provided by cloud environments. Clouds combine high performance and cost-efficiency and are therefore very much sought after. 

The set-up of cloud environment can be quite tedious – fortunately, the needed infrastructure installations are often similar across projects and therefore tools that enable automated infrastructure installations can be used to minimize the manual workload.  The following blog post covers setting up a cloud environment using Ansible, which is one such program.

We will talk about UNIX based system as:

  • most people already have a working knowledge of Windows based systems, but are much less knowledgeable in terms of UNIX
  • most UNIX based systems are open-source under free licenses, so they are cheaper to run in general
  • some handy products (e.g. RStudio server) are only UNIX based so a situation where basic knowledge is necessary can arise

Continue reading