Python

A 3-post collection

PyCharm suppress inspections list

I've been searching for a complete reference to PyCharm's #noinspection markup for some time. I found #noinspection PyUnresolvedReferences for suppressing the warning about an unused import statement with Google, and I was able to guess at PyProtectedMember myself, but it was clear that many remained.

Unfortunately the tooltip action in PyCharm only gives the option to suppress some warnings, revealing for instance the #noinspection PyBroadException markup but few others.

I decided I had had enough and used the excellent Agent Ransack to search for PyUnresolvedReferences within the installation directory. Once I found it I was able to extract the rest. I believe the following to be a complete list. I've found PyArgumentList to be particularly useful when PyCharm is upset about valid syntax (e.g. the functional syntax for enumerations in Python 3.4: enum.Enum('vehicle', 'car bicycle truck') or modules with opaque functions).

PyAbstractClass
PyArgumentEqualDefault
PyArgumentList
PyAssignmentToLoopOrWithParameter
PyAttributeOutsideInit
PyAugmentAssignment
PyBroadException
PyByteLiteral
PyCallByClass
PyCallingNonCallable
PyChainedComparisons
PyClassHasNoInit
PyClassicStyleClass
PyComparisonWithNone
PyCompatibility
PyDecorator
PyDefaultArgument
PyDeprecatedModules
PyDeprecation
PyDictCreation
PyDictDuplicateKeys
PyDocstring
PyDocstringTypes
PyExceptClausesOrder
PyExceptionInherit
PyFromFutureImport
PyGlobalUndefined
PyInconsistentIndentation
PyInitNewSignature
PyInterpreter
PyListCreation
PyMandatoryEncoding
PyMethodFirstArgAssignment
PyMethodMayBeStatic
PyMethodOverriding
PyMethodParameters
PyMissingConstructor
PyNestedDecorators
PyNonAsciiChar
PyNoneFunctionAssignment
PyOldStyleClasses
PyPackageRequirements
PyPep8
PyPep8Naming
PyPropertyAccess
PyPropertyDefinition
PyProtectedMember
PyRaisingNewStyleClass
PyRedeclaration
PyRedundantParentheses
PyReturnFromInit
PySetFunctionToLiteral
PyShadowingBuiltins
PyShadowingNames
PySimplifyBooleanCheck
PySingleQuotedDocstring
PyStatementEffect
PyStringException
PyStringFormat
PySuperArguments
PyTrailingSemicolon
PyTupleAssignmentBalance
PyTupleItemAssignment
PyTypeChecker
PyUnboundLocalVariable
PyUnnecessaryBackslash
PyUnreachableCode
PyUnresolvedReferences
PyUnsupportedFeatures
PyUnusedLocal

Discussion

Deploying Django 1.7 (Python 3) on Ubuntu 14.04

Since the DigitalOcean guide is out of date, here's a new one as short as possible using the default sqlite instead of postgres for simplicity.

Patch:

$ sudo apt-get update
$ sudo apt-get upgrade

We'll use pyvenv-3.4 because virtualenv is for Python 2.
It was shipped in Ubuntu 14.04 with a broken pip, so we'll repair that as well as create our virtual environment at /opt/djenv:

$ pyvenv-3.4 --without-pip /opt/djenv
$ cd /opt/djenv
$ source bin/activate
(djenv) $ curl https://bootstrap.pypa.io/get-pip.py | python

(deactivate exits the virtual environment and source /opt/djenv/bin/activate enters it)

Install django and gunicorn much like Robin Winslow describes on his blog:

(djenv) $ which pip         # check pip points to the right place
/opt/djenv/bin/pip
(djenv) $ pip install django django-extensions     # install django dependencies
(djenv) $ pip install gunicorn
(djenv) $ pip freeze        # see our installed dependencies
Django==1.7.1
django-extensions==1.4.9
gunicorn==19.1.1
six==1.8.0
(djenv) $ django-admin.py startproject projectname  # create new django project
(djenv) $ pip freeze > projectname/requirements.txt # save dependencies into project

The Django 1.7 tutorials haven't quite caught up, when you deploy you'll need to add 127.0.0.1 or localhost to ALLOWED_HOSTS within settings.py or you'll get a Bad Request (400) error when DEBUG = False, answered in this Stack Overflow question.

ALLOWED_HOSTS = ['127.0.0.1', 'localhost']

The deployed static file setup can also be a little confusing. You'll create a new, empty folder on your webserver named static wherever you'd like - root project folder perhaps, alongside manage.py. Specify the full path to this folder within your settings.py as STATIC_ROOT, and whatever relative URL you desire and will be configuring within nginx as STATIC_URL, e.g.:

STATIC_URL = '/static/'
STATIC_ROOT = '/opt/djenv/projectname/static/'

Whenever you change your actual static files, likely located in paths such as /opt/djenv/projectname/appname/static/appname/ and /opt/djenv/projectname/otherappname/static/otherappname/ you need to re-run the collectstatic command so they'll all be gathered and automatically copied into the folder we created - you'll never do this manually:

(djenv) /opt/djenv/projectname$ ./manage.py collectstatic

And of course you must have nginx configured to serve both django running on gunicorn and static files, something like:

server {
    server_name yourdomainorip.com;

    access_log off;

    location /static/ {
        alias /opt/djenv/static/;
    }

    location / {
            proxy_pass http://127.0.0.1:8001;
            proxy_set_header X-Forwarded-Host $server_name;
            proxy_set_header X-Real-IP $remote_addr;
            add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
    }
}

Launching gunicorn with gunicorn_django is deprecated, instead of gunicorn_django --workers=3 --bind yourdomainorip.com:8001 you'll want to use something like:

(djenv) /opt/djenv/projectname$ gunicorn --env DJANGO_SETTINGS_MODULE=projectname.settings projectname.wsgi:application --bind 127.0.0.1:8001

Finally you can configure gunicorn to launch automatically with an Upstart script /etc/init/gunicorn.conf:

start on startup

script
  . /opt/djenv/bin/activate
  cd /opt/djenv/projectname
  gunicorn --env DJANGO_SETTINGS_MODULE=projectname.settings projectname.wsgi:application --bind 127.0.0.1:8001
end script

You can also control it via command-line e.g.: start gunicorn stop gunicorn restart gunicorn.

Discussion

Recovering contacts in Windows Live Mail via harvesting your inbox

So you upgraded a machine from Vista or some such to Windows 7, and lost the email program as informed. You were wise and exported all of the emails and contacts beforehand. So you install Windows Live Mail and hurry through the import procedure. It looks good! Oh maybe you delete the backup, but just into the Recycle Bin.

Later you're tinkering around, deleting and moving files. It's not your machine and there's already so much trash in the bin, you wouldn't want your trash mixed with their trash (the horror!), so you empty yours. Whoops there went the backup, but all was imported perfectly so no worry.

Or was it! Actually the emails survived all right, only missing some images attached to a letter in the Drafts folder. Probably saved in a temporary location that wasn't preserved in the Windows 7 upgrade. The contacts though, are all horribly terribly wrong. Important people are missing! You were fooled into thinking a first and last name on your emails implied an association with their profile in your contacts, but alas it is not so!

Ah, but you are a programmer. Bahahaha!

So with time of the essence I coded up some quick and dirty Python.

Somewhat dumbed down harvest instructions

  1. First create the folder C:\Mail.
  2. In Windows Live Mail, press Alt+F for the File menu, then Export → Messages… → Microsoft Windows Live Mail
  3. Browse to C:\Mail, and complete the export, yes to All folders. Afterward you've a directory full of emails in .eml format.
  4. Download and install Python 2.6.4. Choose Windows x86 MSI Installer if you're not sure which. Install it to the default directory: C:\Python26
  5. Download this Python script (displayed above). Right click that download link and Save link as… C:\Mail\eml_to_csv.py
  6. Open a command prompt (Windows Key + R → cmd enter).
  7. In the console type:
    C:\Python26\python.exe C:\Mail\eml_to_csv.py C:\Mail C:\Mail\contacts.csv. After completion you now have a contacts file which can be imported.
  8. Back in Windows Live Mail in Contacts go Alt+F → Import → Comma Separated Values and select C:\Mail\contacts.csv
  9. You should see over a dozen separate mappings, First Name to First Name, Last Name to Last Name and so on, if you don't then stop here. If you do, click Finish! :)
  10. Purge any unwanted junk contacts! Use the information I put under Notes to determine the most recent email the contact was harvested from. Use these dates to your advantage to distinguish old/new addresses and so forth.

For troubleshooting/badasses

  1. After step 7 open the .csv file in Microsoft Excel.
  2. Choose Delimited, on the next page uncheck Tab and check Comma, then Finish.
  3. Use your awesome spreadsheet knowledge to sort (not the first line though!), fix, and purge your contacts. (Just FYI if you've no name with a contact I recommend clearing the Name field rather than placing the e-mail address here. Windows Live Mail will nicely distinguish these contacts as nameless then.)
  4. Upon completion use Save As… CSV (Comma delimited) and if asked if you want to keep this format, choose Yes. It won't be enough to use plain Save.
  5. Continue at step 8 and import your new and improved .csv file into Windows Live Mail instead.

If you want to be really cool, after Excel processing, open the file in a text editor and replace tabs with newlines for a prettier display of Notes in Windows Live Mail. Don't do this before Excel or try to open this new file in Excel afterward, as the newlines will confuse its CSV parser. (They are however valid, and Windows Live Mail will take the file just fine.)

On top of that, if you know how, you can search within the .eml files yourself for lines containing: Reply-to and correcting any contact name/addresses with these lines. Unfortunately the Python script doesn't yet handle these automatically.

Finally

Hooray! Contacts recovered! At least as many as in your emails. Better than nothing! I showed that machine who's boss. *flexes programming prowess* Next time no backup deleting idiocy. Also, have a backup to begin with, but you already knew that.

Discussion