python-course.eu

Numerical Programming with Python

By Bernd Klein. Last modified: 01 Feb 2022.

Numerical Programming Definition

The term "Numerical Computing" - a.k.a. numerical computing or scientific computing - can be misleading. One can think about it as "having to do with numbers" as opposed to algorithms dealing with texts for example. If you think of Google and the way it provides links to websites for your search inquiries, you may think about the underlying algorithm as a text based one. Yet, the core of the Google search engine is numerical. To perform the PageRank algorithm Google executes the world's largest matrix computation.

Art of Formulas

Numerical Computing defines an area of computer science and mathematics dealing with algorithms for numerical approximations of problems from mathematical or numerical analysis, in other words: Algorithms solving problems involving continuous variables. Numerical analysis is used to solve science and engineering problems.

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Data Science and Data Analysis

This tutorial can be used as an online course on Numerical Python as it is needed by Data Scientists and Data Analysts.

Data science is an interdisciplinary subject which includes for example statistics and computer science, especially programming and problem solving skills. Data Science includes everything which is necessary to create and prepare data, to manipulate, filter and clense data and to analyse data. Data can be both structured and unstructured. We could also say Data Science includes all the techniques needed to extract and gain information and insight from data.

Data Science is an umpbrella term which incorporates data analysis, statistics, machine learning and other related scientific fields in order to understand and analyze data.

Another term occuring quite often in this context is "Big Data". Big Data is for sure one of the most often used buzzwords in the software-related marketing world. Marketing managers have found out that using this term can boost the sales of their products, regardless of the fact if they are really dealing with big data or not. The term is often used in fuzzy ways.

Big data is data which is too large and complex, so that it is hard for data-processing application software to deal with them. The problems include capturing and collecting data, data storage, search the data, visualization of the data, querying, and so on.

The following concepts are associated with big data:

The big question is how useful Python is for these purposes. If we would only use Python without any special modules, this language could only poorly perform on the previously mentioned tasks. We will describe the necessary tools in the following chapter.

Connections between Python, Numpy, Matplotlib, Scipy and Pandas

Overview diagram: Comparison between Python, NumPy, SciPy, Matplotlib and Pandas

Python is a general-purpose language and as such it can and it is widely used by system administrators for operating system administration, by web developpers as a tool to create dynamic websites and by linguists for natural language processing tasks. Being a truely general-purpose language, Python can of course - without using any special numerical modules - be used to solve numerical problems as well. So far so good, but the crux of the matter is the execution speed.

Pure Python without any numerical modules couldn't be used for numerical tasks Matlab, R and other languages are designed for. If it comes to computational problem solving, it is of greatest importance to consider the performance of algorithms, both concerning speed and data usage. If we use Python in combination with its modules NumPy, SciPy, Matplotlib and Pandas, it belongs to the top numerical programming languages. It is as efficient - if not even more efficient - than Matlab or R.

Numpy is a module which provides the basic data structures, implementing multi-dimensional arrays and matrices. Besides that the module supplies the necessary functionalities to create and manipulate these data structures.

SciPy is based on top of Numpy, i.e. it uses the data structures provided by NumPy. It extends the capabilities of NumPy with further useful functions for minimization, regression, Fourier-transformation and many others.

Matplotlib is a plotting library for the Python programming language and the numerically oriented modules like NumPy and SciPy. The youngest child in this family of modules is Pandas.

Pandas is using all of the previously mentioned modules. It's build on top of them to provide a module for the Python language, which is also capable of data manipulation and analysis. The special focus of Pandas consists in offering data structures and operations for manipulating numerical tables and time series. The name is derived from the term "panel data". Pandas is well suited for working with tabular data as it is known from spread sheet programming like Excel.

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Upcoming online Courses

Enrol here

Python, An Alternative to Matlab

Python is becoming more and more the main programming language for data scientists. Yet, there are still many scientists and engineers in the scientific and engineering world that use R and MATLAB to solve their data analysis and data science problems. It's a question troubling lots of people, which language they should choose:

The functionality of R was developed with statisticians in mind, whereas Python is a general-purpose language. Nevertheless, Python is also - in combination with its specialized modules, like Numpy, Scipy, Matplotlib, Pandas and so, - an ideal programming language for solving numerical problems. Furthermore, the community of Python is a lot larger and faster growing than the one from R.

The principal disadvantage of MATLAB against Python are the costs. Python with NumPy, SciPy, Matplotlib and Pandas is completely free, whereas MATLAB can be very expensive. "Free" means both "free" as in "free beer" and "free" as in "freedom"!

Even though MATLAB has a huge number of additional toolboxes available, Python has the advantage that it is a more modern and complete programming language. Python is continually becoming more powerful by a rapidly growing number of specialized modules.

Python in combination with Numpy, Scipy, Matplotlib and Pandas can be used as a complete replacement for MATLAB.

Equivalence between Python, Numpy, Scipy, Matplotlib and Matlab

Live Python training

instructor-led training course

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Upcoming online Courses

Enrol here

In this chapter

Page
Description
1. Introduction to NumPyNumPy tutorial: NumPy is used for scientific computing with Python. This is an introduction for beginners with examples.
2. Creating Numpy ArraysNumPy tutorial: Creating basic array structures and manipulating arrays. Introducing shape, dimension and Slicing. One-dimensional and multi-simensional arrays.
3. Numpy Data Objects, dtypeTutorial and Introduction to data types and dtype in numpy
4. Numerical Operations on Numpy ArraysUsing operators in numpy and applying them on Numpy Arrays, compared to using operators in 'pure' Python
5. Numpy Arrays: Concatenating, Flattening and Adding DimensionsNumPy: Changing the Dimensions of arrays with the functions newaxis, reshape and ravel. concatenating arrays
6. Python, Random Numbers and ProbabilityPython, Numpy and ways to cope with probability and randomness. Introductory Tutorial.
7. Weighted ProbabilitiesPython Tutorial on weighted random Choice and Sample. Synthetically created Sales Figures. Exercises with solutions.
8. Synthetical Test Data With PythonCreating Synthetical Test and Learn Sets using random functions of Python and Numpy
9. Numpy: Boolean IndexingNumPy: Boolean Masking of Arrays
10. Matrix Arithmetics under NumPy and PythonIntroduction with examples into Matrix-Arithmetics with the NumPy Module
11. Reading and Writing Data Files: ndarraysTutorial on how to read and write from and to files.
12. Overview of MatplotlibIntroduction to Matplotlib for beginners as part of a complete Python tutorial.
13. Formatting a plot in MatplotlibIntroduction into Matplotlib for beginners as part of a complete Python tutorial.
14. Matplotlib Object HierarchyIntroduction in the Hierarchy of Matplotlib object.
15. Spines and Ticks in MatplotlibMatplotlib Tutorial: Customizing spines and ticks
16. Adding Legends and Annotations in MatplotlibMatplotlib Tutorial: Adding legends and annotations to plots.
17. Creating Subplots in MatplotlibMatplotlib Tutorial: Subplots and their creation.
18. Gridspec in MatplotlibMatplotlib Tutorial: Introduction into Gridspecs and examples
19. Histograms with MatplotlibPython-Course: Creating Histograms with Python and Matplotlib.
20. Contour Plots with MatplotlibMatplotlib Tutorial: Introduction in Countour Plots both line based and filled contour plots.
21. Image Processing in Python with MatplotlibImage processing in Python with Matplotlib, Numpy and Scipy. Tutorial with examples.
22. Image Processing Techniques with Python and MatplotlibTutorial on Image Processing Techniques with Python, Numpy and Matplotlib
23. Creating Videos from One or More ImagesIn this Python course we show how to create a video from one or more images. Simply by showing the images for some seconds sequentially or suing a tiling technique.
24. Moving Watermarks Video with PythonThis Python tutorial explains how to create videos with moving object (visible watermarks) from one or more picture by using Numpy and Matplotlib.
25. Introduction to PandasTutorial with Examples on Pandas in Python
26. Pandas DataFrameTutorial on DataFrame data type of Pandas
27. Accessing and Changing values of DataFramesMatplotlib Tutorial: replace, at, loc to change values.
28. Pandas PivotIntroduction in the Pandas pivot function
29. Pandas: groupbyPandas Tutorial: split, apply and combine with groupby.
30. Pandas Groupby ExampleAn extensive Pandas example using groupby to evaluate a csv data file.
31. Reading and Writing Data in PandasTutorial with Examples on Reading and Writing Data in Pandas
32. Dealing with NaNDealing with NaN (Not a Number) values in Python and in Numpy
33. Binning in Python and PandasBinning data with Python functionalities and by using Pandas binning possibilities
34. Multi-level Indexing in PandasContinuation of the Pandas Tutorial. Introducing details of indexing, especially multi level indexing
35. Data Visualization with PandasVisualisations with Pandas and Python
36. Python Date and TimePython, Time and Date
37. Time Series in Pandas and PythonPython, Pandas and Time Series
38. Expenses and income example with Pandas and PythonExpenses, income and taxes example with Pandas and Python
39. Net Income Method Example with Numpy, Matplotlib and ScipyExtensive Example with Panadas: Net Income Taxes calculations
40. Estimation of Corona cases with Python and PandasEstimating the number of infected people, the real number i.e. known and unknown in Germany
41. Linear Combinations in PythonCalculating linear combinations for finite sets of terms and constants in Python