Python Development Setup

October 2, 2016

python | git | django |

Notes, Documentation, Code Snippets

Table of Contents

Getting Set Up

I basically just followed this guide for the most part. Essentially, install Homebrew and XCode and create a .bash_profile.

Next I followed this guide kinda. Install both 2.x and 3.x pythons using Homebrew, and install pip, virtualenv, virtualenvwrapper if not already installed and add a line to .bash_profile to only allow running pip in virtual environments (but also create an alias gpip that allows you to install / upgrade globally).


Homebrew

Python, Git and other packages are best installed via Homebrew.

# Update Homebrew
brew update

# See which packages are outdated
brew outdated

# Upgrade all packages
brew upgrade

# Install a package
brew install git

# Check for problems
brew doctor

Python Project Setup

Before starting a project, I would recommend:

  1. Updating Homebrew and any packages that are outdated
  2. Updating Pip and any packages that are outdated

# Create virtualenv project with Python3
mkproject -p python3 test_project

# or if the project folder already exists
mkvirtualenv -p python3 test_project

# Install Django into virtualenv
pip install django

Pip

{% highlight sh %}

Upgrade

pip install -U pip gpip install -U pip # in my case due to my .bash_profile

Search packages

pip search django

Install

pip install django pip install django==1.8.1

Upgrade

pip install -U django

Uninstall

pip uninstall django

Show info about package

pip show django

Show what’s installed in current virtualenv

pip list

Show what’s outdated

pip list –outdated

Dump current set up to file

pip freeze > requirements.txt

Install based on a requirements file

pip install -r requirements.txt {% endhighlight %}


Git and GitHub

Resources

Basic Commands

{% highlight sh %}

To install

brew install git

Pull down and clone an existing repository

git clone https://github.com/getup8/basic.git

Initialize a local folder as a repository

Add files to staging

Commit the files or changes with a useful description message

git init git add [filenames or –all] git commit -m “Useful commit description”

Skip the staging area and just commit new changes (files need to all be

trackd already)

git commit -a -m “added new benchmarks”

Add a remote repository and push local commit to it

git remote add origin https://github.com/getup8/nano-project-1.git git push -u origin master

Get the status of files in the repository

git status

See the changes made since last commit that are not yet staged

git diff

See the changes that are staged

git diff –staged

Forget what this does differently

git diff –cached

Remove file from tracking:

git rm PROJECTS.md

Remove file from staging but not from filesystem:

git rm –cached README git mv README.md README

See commit history of repository:

git log

See commit diffs and limit to last two entries

git log -p -2

Other commands

git log –stat git log –pretty=oneline|short|full|fuller git log –pretty=format:“%h %s” –graph git log –since=2.weeks

Add a file you forgot

git commit –amend

Remove from staging

git reset HEAD CONTRIBUTING.md

Totally reset everything (throw away local commits) like you’re cloning

from remote

git reset –hard [HEAD]

Revert back to initial version of file before changes

git checkout – CONTRIBUTING.md

Revert all files

git checkout – .

Remove files that are not tracked (I think)

git clean -df

See list of current remote repositories

git remote -v

Add a remote repository

git remote add [shortname] [url] git remote add pb https://github.com/paulboone/ticgit

Need more documentation here

git fetch pb git fetch origin

Sync local repo with remote

get pull

Another way to sync with remote I believe

git fetch origin git merge origin/master

Push to remote (what does -u do?)

git push origin master

Get more info on remote

git remote show origin {% endhighlight %}


Markdown

GitHub Markdown Docs


Jekyll

I’m using Jekyll to make writing notes and documentation easier for my GitHub projects and other ventures. Below are some quick resources.

Resources

Quick Commands

{% highlight sh %}

After making a change in the project run this to build into _site directory

jekyll build

Build the project into _site and watch and auto-build after changes

jekyll build –watch

Firing up the local server http://localhost:4000

bundle exec jekyll serve {% endhighlight %}

More to come..


Sass

Resources

Installation is as easy as: sudo gem install sass

Example Usage

Some simple documentation of key concepts below. The code is written in SCSS not SASS, the main difference being SASS doesn’t use semi-colons or parens.

Variables

{% highlight scss %} $font-stack: Arial, sans-serif; $color: #CCCCCC; {% endhighlight %}

Nesting

{% highlight scss %} nav { ul { } } {% endhighlight %}

Partials

Write snippets in separate files to include in other Sass files: _partial.scss Used with the @import directive: @import ‘partial’;

Mixins

Kinda like functions, useful especially for vendor prefixes. {% highlight scss %} @mixin border-radius($radius) { -webkit-border-radius: $radius; -moz-border-radius: $radius; -ms-border-radius: $radius;
border-radius: $radius; }

.box { @include border-radius(10px); } {% endhighlight %}

Extend/Inheritance

Share CSS between selectors. Inherit from other selectors. {% highlight scss %} .message { border: 1px solid #ccc; padding: 10px; color: #333; }

.success { @extend .message; border-color: green; } {% endhighlight %}

Operators / Math

Multiply, divide, add, subtract, modulus. {% highlight scss %} .container { width: 100%; }

article[role=“main”] { float: left; width: 600px / 960px * 100%; }

aside[role=“complimentary”] { float: right; width: 300px / 960px * 100%; } {% endhighlight %}

Running Sass

To run Sass from the command line, just use: sass input.scss output.css

You can also tell Sass to watch the file and update the CSS every time the Sass file changes: sass --watch input.scss:output.css

If you have a directory with many Sass files, you can also tell Sass to watch the entire directory: sass --watch app/sass:public/stylesheets

Further Reading


Getting Started on Heroku

Following the steps outlined here.

Initial assumptions

  • Python installed
  • virtualenv, setuptools, pip installed
  • Heroku account
  • Heroku toolkit installed (Heroku client, git, Foreman - an easy way to run apps locally)

Steps

Clone the getting started repository and move into the directory. This initializes git and sets up some boilerplate structure and files. {% highlight sh %} git clone https://github.com/heroku/python-getting-started.git cd python-getting-started {% endhighlight %}

Login to Heroku and create an app on Heroku, which also creates a git “remote” which is associated with the local git repository. {% highlight sh %} heroku login heroku create {% endhighlight %}

Deploy the code to Heroku. {% highlight sh %} git push heroku master {% endhighlight %}

Allocate one dyno (lightweight linux “container”) to the app. {% highlight sh %} heroku ps:scale web=1 {% endhighlight %}

Open the app to check it out. {% highlight sh %} heroku open

https://aqueous-cove-8924.herokuapp.com/

{% endhighlight %}

View server logs. {% highlight sh %} heroku logs –tail {% endhighlight %}

Procfiles define the processes that run and the command that is executed to start your app.

To add more containers/dynos, you can scale up to make the app persistently “up”. You only get 750 free dyno hours though so this would make our app not free after a short time. {% highlight sh %} heroku ps:scale web=2 {% endhighlight %}

Declaring app dependencies is done via requirements.txt. Heroku reads this file and installs the proper dependencies via pip. To do this locally, we should use a virtual env. {% highlight sh %} virtualenv venv source venv/bin/activate {% endhighlight %}

Run pip to install these dependencies locally (Heroku does the same). Doing this lets us run the app locally, which is really what we want to do in development. {% highlight sh %} pip install -r requirements.txt –allow-all-external {% endhighlight %}

Running foreman reads the Procfile (similar to Heroku) and runs the app: {% highlight sh %} foreman start web

http://0.0.0.0:5000

{% endhighlight %}

In order to modify anything (let’s modify the Procfile for instance to require another plugin), re-run these commands. I’m guessing pip isn’t necessary unless you need something else installed. {% highlight sh %} pip install -r requirements.txt –allow-all-external foreman start {% endhighlight %}

Now we deploy these changes to Heroku. First add the modified files to git and then commit the changes to the local repository. {% highlight sh %} git add . git commit -m “Demo” {% endhighlight %}

Then just deploy to Heroku again. {% highlight sh %} git push heroku master heroku open {% endhighlight %}

There are tons of addons for Heroku as well, should be checked out at some point.

To run a REPL (read eval print loop) / shell interface, use the run command. {% highlight sh %} heroku run python manage.py shell heroku run bash

Type “exit” to exit the shell and release the dyno.

{% endhighlight %}

Check out config vars later, maybe good for dev/prod and other things?

Databases are addons, you can see the current ones by just running: {% highlight sh %} heroku addons {% endhighlight %} I think they set up postgres automatically.

You can also run the following to see the current config vars set up (relevant to set up database): {% highlight sh %} heroku config # General config heroku pg # Specifically for Postgres {% endhighlight %}

I guess this app is already configured with a database and you can visit (what? i think just a view) by just appending /db after the URL: https://aqueous-cove-8924.herokuapp.com/db

But initially it’ll give an error b/c the tables don’t exist. Need to initialize it first, the Django way. {% highlight sh %} heroku run python manage.py syncdb {% endhighlight %} It creates the tables and now visiting that URL/view works.

You can access the DB directly (assuming postgres is installed locally) via: {% highlight sh %} heroku pg:psql \dt # to show tables in postgres {% endhighlight %}

Rename the app {% highlight sh %} heroku apps:rename {% endhighlight %}

Further Reading:


VirtualEnv

Resources

I installed all of those.

Before doing anything anymore: {% highlight sh %}

Create the project, virtualenv, move into the dir and activate

mkproject dt

Delete

rmvirtualenv dt

Show current environments

lsvirtualenv

To get out of the venv

deactivate

To move to directory and activate an existing virtualenv

workon dt {% endhighlight %}


Vagrant

Resources

If you’d like to see what was installed in the VM, look in /vagrant/pg_config.sh.

{% highlight sh %} vagrant up vagrant ssh exit # exit the command line vagrant halt {% endhighlight %}

Random: To connect psql to the forum database for Lesson 3, type psql forum at the command line. To exit psql, type \q or Control-D (^D).


Postgres

Resources

Commands

{% highlight sh %} – Connect to database psql [db] {% endhighlight %}

{% highlight psql %} – Show databases \l \list \l+

– Connect to database \c [database_name]

– Show tables \dt

– Desc table \d [tablename]

– Quit \q

– Help - get all these commands \h

– Connectinon info \conninfo

– Change database name ALTER DATABASE name RENAME TO new_name

{% endhighlight %}

http://www.postgresql.org/docs/current/interactive/dblink.html

{% highlight psql %}

CREATE EXTENSION dblink;

– Verify SELECT pg_namespace.nspname, pg_proc.proname FROM pg_proc, pg_namespace WHERE pg_proc.pronamespace=pg_namespace.oid AND pg_proc.proname LIKE ‘%dblink%’;

– Test connection SELECT dblink_connect(‘host=localhost user=alexscott dbname=wewander_db’);

– Add back the social login set up info UPDATE django_site SET domain = ‘localhost:8000’, name = ‘localhost’ where id = 1;

INSERT INTO socialaccount_socialapp SELECT * FROM dblink(‘dbname=wewander_db2’, ‘SELECT * FROM socialaccount_socialapp’) AS t1(id integer, provider character varying(30), name character varying(40), client_id character varying(191), secret character varying(191), key character varying(191));

INSERT INTO socialaccount_socialapp_sites (socialapp_id, site_id) VALUES (1, 1), (2, 1), (3, 1);

{% endhighlight %}

{% highlight psql %} CREATE DATABASE db DROP DATABASE db DROP TABLE table CREATE VIEW view AS SELECT … – Useful for aggregations {% endhighlight %}

Database Concepts

  • Atomicity: a command executes as a whole or not at all (commit!)
  • Normalized vs Denormalized: Normalized databases have relationships between tables that are really there among the data.

SQLAlchemy

Resources


Protocols

Protocols are the “gramatical rules” that ensure that clients and servers (and really any computer on the internet) are communicating in the same way.

  • Transmission Control Protocol (TCP) Enables info to be broken into small packets and sent. Counterpart is UDP (User Data Protocol) which is good for streaming
  • Internet Protocol (IP) Like a postal address. Can be static or dynamic. Computer looks up IP in a DNS (Domain Name Server) Uses the address to initiate communication Operating systems use Ports to designate channels of communication on the same IP address. Can range from 0 to 65,536. First 10K often reserved by OS. 80 is most common. 8080 is often used for web communication. When client and server on the same machine, we use “localhost” which has the IP address 127.0.0.1
  • Hypertext Transfer Protocol (HTTP, HTTP2?) Clients tell servers what they want using HTTP “verbs” Most common are GET and POST requests GETs are sometimes called safe methods Server replies to these requests with a Status Code. Can also supply requested resources like HTML, CSS, JS, images, etc 200: Successful GET 301: Successful POST 404: File not found
Alex Scott If you're a nerd, I'm a nerd.
comments powered by Disqus