OUTDATED: Technology Training - Python & FutureSystems

… warning:: this section will be updated and replaced.

This section is meant to give an overview of the python tools needed for doing for this course.

These are really powerful tools which every data scientist who wishes to use python must know.

NumPy - It is popular library on top of which many other libraries (like pandas, scipy) are built. It provides a way a vectorizing data. This helps to organize in a more intuitive fashion and also helps us use the various matrix operations which are popularly used by the machine learning community. Matplotlib: This a data visualization package. It allows you to create graphs charts and other such diagrams. It supports Images in JPEG, GIF, TIFF format. SciPy: SciPy is a library built above numpy and has a number of off the shelf algorithms / operations implemented. These include algorithms from calculus(like integration), statistics, linear algebra, image-processing, signal processing, machine learning, etc.

Python for Big Data: NumPy, SciPy, MatPlotlib

This section is meant to give an overview of the python tools needed for doing for this course. These are really powerful tools which every data scientist who wishes to use python must know.

Introduction

This section is meant to give an overview of the python tools needed for doing for this course. These are really powerful tools which every data scientist who wishes to use python must know. This section covers NumPy, MatPlotLib, and Scipy.

Pycharm

is an Integrated Development Environment (IDE) used for programming in Python. It provides code analysis, a graphical debugger, an integrated unit tester, integration with git.

Video: 8:56: Pycharm: https://youtu.be/X8ZpbZweJcw

Python in 45 minutes

Here is an introductory video about the Python programming language that we found on the internet. Naturally there are many alternatives to this video, but the video is probably a good start. It also uses PyCharm which we recommend.

https://www.youtube.com/watch?v=N4mEzFDjqtA

How much you want to understand of python is actually a bit up to your, while its goot to know classes and inheritance, you may be able for this class to get away without using it. However, we do recommend that you learn it.

Numpy 1

NumPy - It is popular library on top of which many other libraries (like pandas, scipy) are built. It provides a way a vectorizing data. This helps to organize in a more intuitive fashion and also helps us use the various matrix operations which are popularly used by the machine learning community.

Video 1: 11:53: Numpy 1: http://youtu.be/mN_JpGO9Y6s

Video 2: 11:26: Numpy 2: http://youtu.be/7QfW7AT7UNU

Video 3: 11:51: Numpy 3: http://youtu.be/Ccb67Q5gpsk

Matplotlib

Matplotlib is a data visualization package. It allows you to create graphs charts and other such diagrams. It supports Images in JPEG, GIF, TIFF format.

Video 1: 11:21: Matplotlib 1: http://youtu.be/3UOvB5OmtYE

Video 2: 8:32: Matplotlib 2: http://youtu.be/9ONSnsN4hcg

Scipy 1

SciPy is a library built above numpy and has a number of off the shelf algorithms / operations implemented. These include algorithms from calculus(like integration), statistics, linear algebra, image-processing, signal processing, machine learning, etc.

Video 1: 6:57: Scipy 1: http://youtu.be/lpC6Mn-09jY

Video 2: 8:52: http://youtu.be/-XKBz7qCUqw

OUTDATED: Using FutureSystems (Please do not do yet)

This section is meant to give an overview of the FutureSystems and how to use for the Big Data Course. In addition to this creating FutureSystems Account, Uploading OpenId and SSH Key and how to instantiate and log into Virtual Machine and accessing Ipython are covered. In the end we discuss about running Python and Java on Virtual Machine.

FutureSystems Overview

In this video we introduce FutureSystems in terms of its services and features.

FirstProgram.java: http://openedx.scholargrid.org:18010/c4x/SoIC/INFO-I-523/asset/FirstProgram.java

Video: 12:12: FutureGid: http://youtu.be/RibpNSyd4qg

Creating Portal Account

This lesson explains how to create a portal account, which is the first step in gaining access to FutureSystems.

See Lesson 4 and 7 for SSH key generation on Linux, OSX or Windows.

Video: 11:50: FutureGrid Introduction: http://youtu.be/X6zeVEALzTk

OUTDATED: Upload an OpenId

This lesson explains how to upload and use OpenID to easily log into the FutureSystems portal.

Video: 1:34: Upload an OpenID: http://youtu.be/rZzpCYWDEpI

SSH Key Generation using ssh-keygen command

SSH keys are used to identify user accounts in most systems including FutureSystems. This lesson walks you through generating an SSH key via ssh-keygen command line tool.

Video: 4:06: ssh-key gen: http://youtu.be/pQb2VV1zNIc

Shell Access via SSH

This lesson explains how to get access FutureSystems resources vis SSH terminal with your registered SSH key.

Video: 2:34 Shell Access via SSH: http://youtu.be/aJDXfvOrzRE

Advanced SSH

This lesson shows you how to write SSH ‘config’ file in advanced settings.

Video: 2:47: Advanced SSH: http://youtu.be/eYanElmtqMo

SSH Key Generation via putty (Windows user only)

This lesson is for Windows users.

You will learn how to create an SSH key using PuTTYgen, add the public key to you FutureSystems portal, and then login using the PuTTY SSH client.

Video: 3:51: Windows users: http://youtu.be/irmVJKwWQCU

Using FS - Creating VM using Cloudmesh and running IPython

This lesson explains how to log into FutureSystems and our customized shell and menu options that will simplify management of the VMs for this upcoming lessons.

Instruction is at: http://cloudmesh.github.io/introduction_to_cloud_computing/class/cm-mooc/cm-mooc.html

Video: 19:28: Using FS - Creating VM using Cloudmesh and running IPython: http://youtu.be/nbZbJxheLwc