Difference between revisions of "Python Resources"

From Simson Garfinkel
Jump to navigationJump to search
 
(43 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Python Source Code and Modules==
I've reviewed these resources and highly recommend them. I periodically update this page, but it may be out of date.
* ipython - a friendly interactive shell for python. [http://ipython.scipy.org/moin/ Info] [http://www.onlamp.com/pub/a/python/2005/01/27/ipython.html ONLamp Article]
* [http://matplotlib.sourceforge.net/ matplotlib] - a handy system for plotting in python.
* numpy - numeric analysis in python (needed for matplotlib)
* [http://labix.org/python-dateutil dateutil]
* [http://www.crystalspace3d.org/ Crystal Space 3D]
* [http://pyx.sourceforge.net/ PyX] - A Python PS and PDF graphing package
* [http://www.pygame.org/ pygame] - a Python game development system
* [http://codespeak.net/icalendar/ iCalendar] package for Python
* [http://www.vpython.org/ VPython] - 3D Programming in Python
* [http://peak.telecommunity.com/DevCenter/EasyInstall PEAK Easy Install]


==Python GUI Options==
==Learning to use Python==
I need to write some programs that use GUI. Python TKinter is available on all versions of Python, but on the Mac it is slow to start and uses XWindows. Here are some options I'm thinking of:
If you want to learn to use python, here are the key skills that you need to know:


* [http://en.wikibooks.org/wiki/Python_Programming/GUI_Programming wikibooks] has a page on Python Program/GUI Programming options
* Basic Python syntax, including the role of whitespace, variables, objects, functions and modules.
* Intermediate Python syntax, including lambdas, list comprehensions, generators, and decorators.
* Building unit tests with unittest or (better) pytest
* Python package management with setuptools and pip.
* Editing python programs with a decent editor, such as pycharm, or an interactive IDE such as spyder or Jupyter notebook. You can also use emacs or vim. Don't use Idle or Notepad++ or nano.


Books on Tkinter:
Do not learn python2.  
* [http://my.safaribooksonline.com/book/programming/python/0130260363/advanced-topics/ch18#X2ludGVybmFsX0ZsYXNoUmVhZGVyP3htbGlkPTAtMTMtMDI2MDM2LTMvNDA5 Chapter 18 of Core Python Programming] discusses GUI Programming with Tkinter
* [http://docstore.mik.ua/orelly/other/python2/ Programming Python, 2nd Edition, Chapter 7 and 8]
* [http://www.reddit.com/r/Python/comments/i9rzj/are_there_any_good_python_3x_books_for_tkinter/ reddit has a page on Python 3.x books for Tkinter]
* [http://effbot.org/tkinterbook/ Introduction to Tkinter] on effbot.org
* [http://gaezipserver.appspot.com/python/manual/python-3.2-docs/library/tkinter.html Python 3.2 tkinter doc]
* [http://www.ferg.org/thinking_in_tkinter/index.html Thinking in Tkinter]
* [http://wiki.python.org/moin/GuiProgramming Python wiki 'GUI Programming in Python']
* [http://docs.python.org/library/tkinter.html Python 2.7 Tkinter Doc]
* [http://www.pythonware.com/library/tkinter/introduction/ Original 'An introduced to Tkinter'] by Fredrik Lundh


Other options:
To learn python you must be able to run Python programs. You have two main options:
* Uses [http://www.fltk.org/ FLTK], the Fast Light Toolkit


==Python Training==
# Although Python is pre-installed on many computers, do not use the pre-installed version, as it is typically out of date. (It's also frequently python2, which is abandoned.) You can instead download and install [https://www.anaconda.com/ the Anaconda Python distribution]. It has all of the packages you need.
I've seen these and highly recommend them:
# Alternatively, you can learn it entirely using web-based Jupyter notebooks.
* [http://video.google.com/videoplay?docid=7760178035196894549 Advanced Python or Understanding Python], Thomas Wouters, Google Tech Talks, February 21, 2007.  


===To Watch===
===If you don't know how to program===
I haven't seen these yet:
If you have never programmed with Python before, you may want to start with DataCamp's Learn python by example:
* [http://pycon.blip.tv/#2032214 PyCon 2009 Videos]
* https://www.learnpython.org, powered by DataCamp. For example, look at the [https://www.learnpython.org/en/String_Formatting String Formatting] lesson
* [http://video.google.com/videoplay?docid=7760178035196894549#docid=-8437473245141752362 Python 101], PyCon 2011
* [https://www.w3schools.com/python/default.asp  W3School's Python Tutorial]
* [http://video.google.com/videoplay?docid=7760178035196894549#docid=9023849479319414382 Introducing Python], 24 minutes of interviews.
; Python 3.0 training:
* [http://www.youtube.com/watch?v=Qi13Hot6IQw Python 3 tutorial - String formatting and conditions]
* [http://www.youtube.com/watch?v=2uO-sJKCNDo Python 3 Tutorial: Variables and Strings]
* [http://www.youtube.com/watch?v=jECazLigEH8 Python 3 Tutorial: 3 - Strings]
* [http:/www.youtube.com/watch?v=v6nD8f-J_XU Python 3 Tutorial: 17 - String Methods]
* [http://www.youtube.com/watch?v=geoly0UNCvA Ira Krakow's Python 3 Tutorial Part 2 (Strings)‏]
* [http://www.youtube.com/watch?v= fc9buLUiqLE Ira Krakow's Python 3 Essential Training --- Splitting and...‏]


===Printed Materials===
I also recommend:
* [http://diveintopython3.org/ Dive Into Python 3] (book)
* [https://www.diveinto.org/python3/ Dive Into Python 3] (book, online for free; a little dated.)
* [https://wiki.python.org/moin/BeginnersGuide/NonProgrammers Python for Non-Programmers]


==Python Installation==
===If you know how to program===
If you are using a Mac, you almost certainly want to ditch Apple's python implementation and instead install your own from:
If you know another programming language (e.g. C, C++, FORTRAN, Java, etc.), you can learn Python in about 2 hours.
* http://www.python.org/download/releases/


If you want everybody to be able to use your GUI, you probably want to write it in Tkinter (Python's Tcl/Tk bindings). If you want to actually enjoy what you are writing, you want to use wxpython.
Start by reading the Python Tutorial:
* [https://docs.python.org/3/tutorial/index.html Python 3 Tutorial]
 
I then recommend reading the documentation:
* [https://docs.python.org/3/ Official Python Documentation]. Just read the documentation. Honestly. It's well-written.
* [http://book.pythontips.com/en/latest/index.html Python Tips] (book, online for free)
* [https://github.com/google/styleguide/blob/gh-pages/pyguide.md Python Google Style Guide].
 
==Using Python==
Recommended packages and tools for different tasks
 
===Developer Tools===
* [https://www.jetbrains.com/pycharm/ PyCharm], free for educational use (if you have a .edu email address)
* Jupyter Notebook (comes with Anaconda; also available https://jupyter.org/ on the web!)
 
===Financial Modeling===
Nice blog entries:
* [https://pbpython.com/amortization-model.html Building a Financial Model with Pandas], looks at mortgage calculator
 
Some books:
* [https://www.amazon.com/Python-Finance-Analyze-Financial-Data/dp/1491945281 Python for Finance: Analyzing Big Financial Data] (O'Reilly)
* [https://www.amazon.com/Financial-Modelling-Python-Shayne-Fletcher/dp/0470987847 Financial Modelling in Python] (Wiley Finance)
 
===Numerical Data Processing===
* numpy and scipy are the standard numerical analysis tools in python.
* pandas is a popular data analysis platform that integrates with numpy. You can make it run on a cluster with Dask.
* Alternatively, you can use Apache Spark and SparkSQL
* tabulate - create tables easily (but see my tytable replacement)
 
===Date Data===
For working with dates, see:
* [http://labix.org/python-dateutil dateutil], included in the Anaconda distribution
* [http://codespeak.net/icalendar/ iCalendar], package for Python, generates calendears
 
===Image Manipulation===
* Use Pillow, the Python2/3 fork of Python Image Library (PIL). 
 
Fun examples:
* http://jeremykun.com/2012/01/01/random-psychedelic-art/
* http://pythonvision.org/basic-tutorial
* http://stackoverflow.com/questions/138250/read-the-rgb-value-of-a-given-pixel-in-python-programatically
 
=== Text Processing ===
See [[Text processing notes]].
 
===Graphing===
* [http://matplotlib.org/ matplotlib] - A well-developed system for plotting 2d and 3d static graphis. (See my [[Short example of using matplotlib within jupyter]])
* holoview - a more modern but less featured system.
* [https://altair-viz.github.io/index.html Altair: Declarative Visualization in Python]
* [https://github.com/geopy/geopy geopy] - geocoding for Python
* Seaborn. (See [https://www.datacamp.com/community/tutorials/seaborn-python-tutorial Datacamp Tutorial])
 
===Easy Graphics===
* Graphics.py: http://mcsp.wartburg.edu/zelle/python/graphics.py http://mcsp.wartburg.edu/zelle/python/ http://mcsp.wartburg.edu/zelle/python/graphics/graphics.pdf
* HTML graphics:  http://anh.cs.luc.edu/python/hands-on/3.1/handsonHtml/graphics.html
* http://plotdevice.io/
* http://www.cc.gatech.edu/classes/AY2013/cs1301_summer/presentations/calico_graphics.pdf
 
===Database Access===
Python contains built-in support for SQLite3.
 
For using MySQL, you'll need a connector. There are many available.
 
Install MySQL connector on anaconda:
    conda install mysql-connector-python
 
Then you can:
    import mysql.connector as mysql
    c = mysql.connect(host=host,database=db,user=user,password=password)
 
I generally prefer PyMySQL, as it's pure-python and has fewer dependencies.
 
===Python GUI Options===
I recommend writing GUI's using a web browser as a front end and a local web server (see next section).
 
If you must write a native app, I recommend using:
* pyqt - Use PyQT5 (it's part of Anaconda)
* [http://www.fltk.org/ FLTK], the Fast Light Toolkit
 
However, I strongly recommend writing your python program as a web-based app (see next section)
 
==Building Web Applications==
You should use a framework that implements Python's [https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface Web Server Gateway Interface (WSGI)] to write web apps. Popular implementations are:
 
* Bottle (easy, and the whole thing runs from a single file)
* Flask (can run on Apache with Passenger, but needs its own domain name)
 
Bottle has an built-in web server for development. For high performance you can combine it with CherryPy or use the Apache wsgi module.
 
 
For more information, see:
* See also https://wiki.python.org/moin/WebFrameworks
* https://github.com/MichaelPereira/webhooks-demo-application
* https://mattcarrier.com/flask-dreamhost-setup/
 
If you are using anything other CGI, you need some way to tell the web server to reload your python program. (You don't need to do this with CGI---it reloads Python and your program every time you serve another request. That's why it's so slow...)
 
===Running WSGI on Dreamhost===
To reload your program on Dreamhost with passenger:
 
    $ touch tmp/restart.txt        # reload your program
 
To reload your program with mod_wsgi:
 
    $ touch wsgi_app.py
 
===Using Passengers on Dreamhost:===
* https://www.byteconsole.com/get-flask-up-and-running-on-a-dreamhost-server-with-passenger/
* https://help.dreamhost.com/hc/en-us/articles/215769578-Passenger-overview
* https://help.dreamhost.com/hc/en-us/articles/216137717-Python-overview
* https://help.dreamhost.com/hc/en-us/articles/115000218612-Installing-a-custom-version-of-Python
* https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3
* https://help.dreamhost.com/hc/en-us/articles/215489338-Installing-virtualenv-and-custom-modules-in-Python
* https://help.dreamhost.com/hc/en-us/articles/115000221112-Using-pip-to-install-Python-modules
 
===Good Tutorials===
Writing a full application with Apache, Bottle and MongoDB:
* https://reachtim.com/articles/BAM-Short-Stack.html
 
=Advanced Python=
* [https://www.amazon.com/Fluent-Python-Concise-Effective-Programming/dp/1491946008 Fluent Python: Clear, Concise, and Effective Programming] (you can get it on Safari)
 
* [https://www.amazon.com/Test-Driven-Development-Python-Selenium-JavaScript/dp/1491958707/ref=pd_sim_14_6/132-3407075-1922137?_encoding=UTF8 Test Driven Development with Python]


On the Mac, you can install matplotlib with this command using MacPorts::
  % sudo port install py-matplotlib py-ipython
[[Category:Resources]]
[[Category:Resources]]

Latest revision as of 17:17, 2 February 2021

I've reviewed these resources and highly recommend them. I periodically update this page, but it may be out of date.

Learning to use Python

If you want to learn to use python, here are the key skills that you need to know:

  • Basic Python syntax, including the role of whitespace, variables, objects, functions and modules.
  • Intermediate Python syntax, including lambdas, list comprehensions, generators, and decorators.
  • Building unit tests with unittest or (better) pytest
  • Python package management with setuptools and pip.
  • Editing python programs with a decent editor, such as pycharm, or an interactive IDE such as spyder or Jupyter notebook. You can also use emacs or vim. Don't use Idle or Notepad++ or nano.

Do not learn python2.

To learn python you must be able to run Python programs. You have two main options:

  1. Although Python is pre-installed on many computers, do not use the pre-installed version, as it is typically out of date. (It's also frequently python2, which is abandoned.) You can instead download and install the Anaconda Python distribution. It has all of the packages you need.
  2. Alternatively, you can learn it entirely using web-based Jupyter notebooks.

If you don't know how to program

If you have never programmed with Python before, you may want to start with DataCamp's Learn python by example:

I also recommend:

If you know how to program

If you know another programming language (e.g. C, C++, FORTRAN, Java, etc.), you can learn Python in about 2 hours.

Start by reading the Python Tutorial:

I then recommend reading the documentation:

Using Python

Recommended packages and tools for different tasks

Developer Tools

  • PyCharm, free for educational use (if you have a .edu email address)
  • Jupyter Notebook (comes with Anaconda; also available https://jupyter.org/ on the web!)

Financial Modeling

Nice blog entries:

Some books:

Numerical Data Processing

  • numpy and scipy are the standard numerical analysis tools in python.
  • pandas is a popular data analysis platform that integrates with numpy. You can make it run on a cluster with Dask.
  • Alternatively, you can use Apache Spark and SparkSQL
  • tabulate - create tables easily (but see my tytable replacement)

Date Data

For working with dates, see:

  • dateutil, included in the Anaconda distribution
  • iCalendar, package for Python, generates calendears

Image Manipulation

  • Use Pillow, the Python2/3 fork of Python Image Library (PIL).

Fun examples:

Text Processing

See Text processing notes.

Graphing

Easy Graphics

Database Access

Python contains built-in support for SQLite3.

For using MySQL, you'll need a connector. There are many available.

Install MySQL connector on anaconda:

   conda install mysql-connector-python

Then you can:

   import mysql.connector as mysql
   c = mysql.connect(host=host,database=db,user=user,password=password)

I generally prefer PyMySQL, as it's pure-python and has fewer dependencies.

Python GUI Options

I recommend writing GUI's using a web browser as a front end and a local web server (see next section).

If you must write a native app, I recommend using:

  • pyqt - Use PyQT5 (it's part of Anaconda)
  • FLTK, the Fast Light Toolkit

However, I strongly recommend writing your python program as a web-based app (see next section)

Building Web Applications

You should use a framework that implements Python's Web Server Gateway Interface (WSGI) to write web apps. Popular implementations are:

  • Bottle (easy, and the whole thing runs from a single file)
  • Flask (can run on Apache with Passenger, but needs its own domain name)

Bottle has an built-in web server for development. For high performance you can combine it with CherryPy or use the Apache wsgi module.


For more information, see:

If you are using anything other CGI, you need some way to tell the web server to reload your python program. (You don't need to do this with CGI---it reloads Python and your program every time you serve another request. That's why it's so slow...)

Running WSGI on Dreamhost

To reload your program on Dreamhost with passenger:

   $ touch tmp/restart.txt         # reload your program

To reload your program with mod_wsgi:

   $ touch wsgi_app.py 

Using Passengers on Dreamhost:

Good Tutorials

Writing a full application with Apache, Bottle and MongoDB:

Advanced Python