Text processing notes - Revision history

Simson at 01:21, 8 March 2018

2018-03-08T01:21:45Z

← Older revision		Revision as of 18:21, 7 March 2018
Line 6:		Line 6:
	* Remember to start with a small extract of the corpus you want to work with. There's no reason waiting 5 minutes to see how something worked if you could have found out in 5 seconds. Work with a few paragraphs of text, not hundreds of pages, at least while you are learning the langugage.		* Remember to start with a small extract of the corpus you want to work with. There's no reason waiting 5 minutes to see how something worked if you could have found out in 5 seconds. Work with a few paragraphs of text, not hundreds of pages, at least while you are learning the langugage.
	* Although it's tempting to work with ipython or Jupyter notebook, be sure to keep extensive notes of everything you try. The easiest way to do this is to save your code in python files that run completely on their own from the command line. These files are typically called scripts or programs.		* Although it's tempting to work with ipython or Jupyter notebook, be sure to keep extensive notes of everything you try. The easiest way to do this is to save your code in python files that run completely on their own from the command line. These files are typically called scripts or programs.


	==For Beginners==		==For Beginners==
Line 19:		Line 18:
	* https://blog.pusher.com/introduction-to-natural-language-processing-with-python/ (25min, includes basic intro to python)		* https://blog.pusher.com/introduction-to-natural-language-processing-with-python/ (25min, includes basic intro to python)
	* https://www.coursera.org/learn/python-text-mining/lecture/AZCCB/basic-natural-language-processing (University of Michigan, course 4 of 5 of the data science certification) [https://www.coursera.org/learn/python-text-mining (course description)]		* https://www.coursera.org/learn/python-text-mining/lecture/AZCCB/basic-natural-language-processing (University of Michigan, course 4 of 5 of the data science certification) [https://www.coursera.org/learn/python-text-mining (course description)]

			== Regular Expressions ==
			Here are two web-based regular expression testers. Be sure that you use the Python version:
			* https://regex101.com
			* https://www.regexplanet.com/advanced/python/index.html

	==More Advanced==		==More Advanced==
	* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56		* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56
	* http://www.akbarian.org/notes/text-mining-nlp-python/		* http://www.akbarian.org/notes/text-mining-nlp-python/

Simson at 14:57, 4 March 2018

2018-03-04T14:57:23Z

← Older revision		Revision as of 07:57, 4 March 2018
Line 14:		Line 14:
	* http://nealcaren.web.unc.edu/an-introduction-to-text-analysis-with-python-part-1/ Very basic, includes a beginner's introduction to python		* http://nealcaren.web.unc.edu/an-introduction-to-text-analysis-with-python-part-1/ Very basic, includes a beginner's introduction to python
	* http://opentechschool.github.io/python-data-intro/ Assumes you already kind of know Python		* http://opentechschool.github.io/python-data-intro/ Assumes you already kind of know Python
	* ~~https~~://www.~~amazon~~.~~com~~/~~Text-Processing-Python-David-Mertz/dp~~/~~0321112547 This~~ book ~~is available [http://gnosis.cx/TPiP/ online] in text form.~~		* http://www.nltk.org/book/ The NLTK book

	Some videos:		Some videos:
	* https://blog.pusher.com/introduction-to-natural-language-processing-with-python/ (25min, includes basic intro to python)		* https://blog.pusher.com/introduction-to-natural-language-processing-with-python/ (25min, includes basic intro to python)
	* https://www.coursera.org/learn/python-text-mining/lecture/AZCCB/basic-natural-language-processing (University of Michigan, course 4 of 5 of the data science certification) [https://www.coursera.org/learn/python-text-mining (course description)]		* https://www.coursera.org/learn/python-text-mining/lecture/AZCCB/basic-natural-language-processing (University of Michigan, course 4 of 5 of the data science certification) [https://www.coursera.org/learn/python-text-mining (course description)]




	==More Advanced==		==More Advanced==
	* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56		* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56
	* http://www.akbarian.org/notes/text-mining-nlp-python/		* http://www.akbarian.org/notes/text-mining-nlp-python/

Simson at 17:20, 3 March 2018

2018-03-03T17:20:53Z

← Older revision		Revision as of 10:20, 3 March 2018
Line 7:		Line 7:
	* Although it's tempting to work with ipython or Jupyter notebook, be sure to keep extensive notes of everything you try. The easiest way to do this is to save your code in python files that run completely on their own from the command line. These files are typically called scripts or programs.		* Although it's tempting to work with ipython or Jupyter notebook, be sure to keep extensive notes of everything you try. The easiest way to do this is to save your code in python files that run completely on their own from the command line. These files are typically called scripts or programs.


			==For Beginners==

	You really want a basic idea about Python. Check out [[Python Resources]].		You really want a basic idea about Python. Check out [[Python Resources]].

	~~These sources look okay~~, ~~but I haven~~'~~t vetted them personally~~:		* http://nealcaren.web.unc.edu/an-introduction-to-text-analysis-with-python-part-1/ Very basic, includes a beginner's introduction to python
			* http://opentechschool.github.io/python-data-intro/ Assumes you already kind of know Python
	* https://www.amazon.com/Text-Processing-Python-David-Mertz/dp/0321112547 This book is available [http://gnosis.cx/TPiP/ online] in text form.		* https://www.amazon.com/Text-Processing-Python-David-Mertz/dp/0321112547 This book is available [http://gnosis.cx/TPiP/ online] in text form.

			Some videos:
			* https://blog.pusher.com/introduction-to-natural-language-processing-with-python/ (25min, includes basic intro to python)
			* https://www.coursera.org/learn/python-text-mining/lecture/AZCCB/basic-natural-language-processing (University of Michigan, course 4 of 5 of the data science certification) [https://www.coursera.org/learn/python-text-mining (course description)]




			==More Advanced==
	* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56		* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56
			* http://www.akbarian.org/notes/text-mining-nlp-python/

Simson: Created page with "Basic rules for people learning text processing: * Use Python3, not Python2. Python3 is the "modern" version for python which has proper support for Unicode. There will be no..."

2018-03-03T17:15:27Z

Created page with "Basic rules for people learning text processing: * Use Python3, not Python2. Python3 is the "modern" version for python which has proper support for Unicode. There will be no..."

New page

Basic rules for people learning text processing:

* Use Python3, not Python2. Python3 is the "modern" version for python which has proper support for Unicode. There will be no more development on Python2, so if you are learning Python2, you'll then need to learn how Python3 is different.
** In Python 3, the <tt>print</tt> statement requires parentheses. So you will see code that looks like this: <tt>print(3)</tt> and not like this: <tt>print 3</tt>
* Use the Natural Language Toolkit (NLTK), because it gives you a lot of power and there are a lot of examples.
* Remember to start with a small extract of the corpus you want to work with. There's no reason waiting 5 minutes to see how something worked if you could have found out in 5 seconds. Work with a few paragraphs of text, not hundreds of pages, at least while you are learning the langugage.
* Although it's tempting to work with ipython or Jupyter notebook, be sure to keep extensive notes of everything you try. The easiest way to do this is to save your code in python files that run completely on their own from the command line. These files are typically called *scripts* or **programs**.

You really want a basic idea about Python. Check out [[Python Resources]].

These sources look okay, but I haven't vetted them personally:
* https://www.amazon.com/Text-Processing-Python-David-Mertz/dp/0321112547 This book is available [http://gnosis.cx/TPiP/ online] in text form.

* https://medium.com/createdd-notes/introduction-to-natural-language-processing-with-python-294988dbae56