This specific tutorial gives you an overview and speaks about the fundamentals of Python.
There are a large number of high level programming languages available, such as C, C++, and Java. The good news is all high level programming languages are very similar to one another. What differs is mostly the syntax, the libraries available and the way we access those libraries. A library is simply a collection of resources and pre-written code that can used while writing a program. If you learn one language well, you can easily learn a new language in just a fraction of time than it took you to learn the first language.
If you are new to programming, Python is really a great place to start. One of the key features of Python is its simplicity, making it the ideal language for beginners to learn. Most programs in Python require considerably fewer lines of code to perform the same task compared to other languages such as C. This leads to fewer programming errors and reduces the development time needed. In addition, Python comes with an extensive collection of third party resources that extends the capabilities of the language. As such, Python can be used for a large variety of tasks, such as for desktop applications, database applications, network programming, game programming and even in mobile development. Last but not the least, Python is a cross platform language, which means that code written for one operating system, such as Windows, will work well on Mac OS or Linux without making any changes to the initial Python code.
For many people, the Python language is easy to fall in love with. Since its first inception in 1991, Python has become one of the most popular dynamic programming languages, along with Perl, Ruby, and others. Python and Ruby have become especially popular in recent years for building websites by utilizing their numerous web frameworks, like Rails (Ruby) and Django (Python). Such languages are often called scripting languages as they can be used to write quick-and-dirty small programs, or scripts. I don’t like the term “scripting language” as it carries a commutation that they cannot be used for building mission-critical software. Among interpreted languages, Python is distinguished by its large and active scientific computing community. Adoption of Python for scientific computing in both industry applications and academic research has increased significantly since the early 2000s. For data analysis and interactive, exploratory computing and data visualization, Python will inevitably draw comparisons with many other domain-specific open source and commercial programming languages and tools in wide use, such as R, MATLAB, SAS, States and others. In recent years, Python’s improved library support (primarily pandas) has made it a strong alternative for data manipulation tasks. Combined with Python’s strength in general purpose programming, it is an excellent choice as a single language for building data-centric applications.
Python as Glue
Part of Python’s success as a scientific computing platform is prominently its ease of integrating C, C++, and FORTRAN code. Most modern computing environments share a similar net of legacy like FORTRAN and C libraries for doing linear algebra, optimization, integration, fast Fourier transforms, and other such algorithms. The same story had held true for many companies and national labs that have used Python to glue together 3o years worth of legacy software. Most programs consist of small portions of code where most of the time is spent, with large amounts of “glue code” that doesn’t run often. In many cases, the execution time of the glue code is insignificant; effort is most fruitfully invested in optimizing the computational bottlenecks, sometimes by moving the code to a lower-level language like C. During the last few years, the Cython project (https://cython.org) has become one of the preferred ways of both creating fast compiled extensions for Python and also interfacing with C and C++ code.
For those who are less familiar with the scientific Python ecosystem and the libraries used throughout the book, I present the following overview of each library.
NumPy, short for Numerical Python, it is the foundational package for scientific computing in Python. The majority of this book will be based on NumPy and libraries built on top of NumPy.
For numerical data, NumPy arrays are a much more efficient way of storing and manipulating data than the other built-in Python.
Pandas provides rich data structures and functions designed to make working with structured data quickly, easily, and expressively. It is, as you will see, one of the critical ingredients enabling Python to be a powerful and productive data analysis environment. The primary object in pandas that will be most defined is the Datarradle, a two-dimensional tabular column-oriented data structure with both row and column labels:
Pandas combine the high performance array-computing features of NumPy with the flodble data manipulation capabilities of spreadsheets and relational databases (such as SQL). It provides sophisticated indexing functionality to make it easy to reshape, slice and dice, perform aggregations, and select subsets of data pandas as the primary tools. For financial users, pandas features rich, high-performance time series functionality and tools well-suited for working with financial data. In fact, I initially designed pandas as an ideal tool for financial data analysis applications. For users of the R language for statistical computing, the DataFrame name will he familiar, as the object was named after the similar R ace result object. They are not the same, however; the functionality provided by data £ razne in R is essentially a strict subset of that provided by the pandas DataFrame. While this is a post about Python, I will occasionally draw comparisons with R as it is one of the most widely-used open source data analysis environments and will be familiar to many readers. The panda’s name itself is derived from panel data, an Econometrics term for multidimensional structured data sets, and Python data analysis itself.
Matplotlib is the most popular Python library for producing plots and other 2D data visualizations. It was originally created by John D. Hunter (JDH) and is now maintained by a large team of developers. It is well-suited for creating plots suitable for publication. It integrates well with IPython (see below), thus providing a comfortable interactive environment for plotting and exploring data. The plots are also interactive as you can zoom in on a section of the plot and pan around the plot using the toolbar in the plot window.
IPython is the component in the standard scientific Python toolset that ties everything together. It provides a robust and productive environment for interactive and exploratory computing. It is an enhanced Python shell designed to accelerate the writing, testing, and debugging of Python code. It is particularly useful for interactively working with data and visualizing data with matplotlib. IPython is usually involved with the majority of my Python work, including running, debugging, and testing code. Aside from the standard terminal-based IPython shell, the project also provides :
SciPy is a collection of packages addressing a number of different standard problem domains in scientific computing.
Here is a sampling of the packages included:
When asked about my standard development environment, we almost always say IPython plus a text editor. We typically write a program and iteratively test and debug each piece of it in IPython. It is also useful to be able to play around with data interactively and visually verify that a particular set of data manipulations is doing the right thing. Libraries like pandas and NumPy are designed to the easy-to-use in the shell. However, some will still prefer to work in an IDE instead of a text editor. They do provide many nice “code intelligence” features like completion or quickly pulling up the documentation associated with functions and classes. Here are some that you can explore:
Get Updates on Tech posts, Interview & Certification questions and training schedules