awesome-math Documentation

A curated list of awesome mathematics resources

15,070 stars Python Python 1 file · ~31,231 tokens #awesome#awesome-list#lecture-notes#list#mathematics
RAW Doc

1. Project Overview & Quickstart (rossant/awesome-math)

Awesome Math [](https://awesome.re)

Awesome Math is a curated list of high-quality resources for learning, teaching, and doing mathematics. It covers pure mathematics, applied mathematics, computation, formal proof, and the use of AI in mathematics.

Most resources are free to access. Paid resources may be included when they are widely respected, unusually useful, and hard to replace. Access limits should be stated in the entry.

Read the contribution guidelines before suggesting a resource.

Contents

Start Here

Learning Platforms

  • Khan Academy - Free, self-paced lessons and exercises from arithmetic through calculus and beyond.
  • Coursera - University and industry mathematics courses, with access and payment varying by course.
  • MIT OpenCourseWare - Free MIT course materials, including notes, assignments, and exams for undergraduate and graduate mathematics.
  • edX - University and professional mathematics courses, with audit and paid options varying by course.
  • Brilliant - Interactive, problem-based lessons in foundational mathematics, probability, and related quantitative subjects.
  • WooTube - Video lessons by Eddie Woo covering secondary-school mathematics with an emphasis on conceptual understanding.
  • Mathispower4u - More than 9,000 free, closed-captioned video lessons and worked examples organized from middle-school mathematics through calculus, linear algebra, differential equations, and statistics, by James Sousa.
  • Mathigon - Interactive courses, activities, and visual explanations for school and early university mathematics.
  • Ximera - Free interactive mathematics textbooks from Ohio State University.
  • Almost Fun - Free interactive middle- and high-school mathematics lessons, practice problems, and teacher resources from a nonprofit organization.
  • Oxford Mathematics - Lectures and public talks from the University of Oxford, including accessible explanations of advanced topics.
  • Math Academy - Paid adaptive platform with diagnostic assessment and mastery-based courses from fourth-grade mathematics through university topics.
  • OSSU Math - Free, prerequisite-ordered self-study curriculum modeled on the mathematics requirements of an undergraduate degree.
  • Waterloo Open Math - Free university-level courseware with lessons, exercises, and immediate feedback from the University of Waterloo.

Proof and Problem Solving

Video Courses and Series

  • Brandon Foltz - Video courses in statistics, probability, and quantitative methods with worked examples.
  • StatQuest - Introductory video explanations of statistics, probability, and machine learning.
  • 3Blue1Brown - Visual explanations of core mathematics, including calculus, linear algebra, and differential equations.
  • NPTEL - University lecture courses from Indian institutes covering mathematics and related technical subjects.
  • Numberphile - Popular mathematics videos featuring mathematicians discussing problems, ideas, and curiosities.
  • PatrickJMT - Worked-example videos for algebra, calculus, and other introductory college topics.
  • Professor Leonard - Full lecture series in precalculus, calculus, and differential equations for college students.
  • MIT OpenCourseWare - MIT course recordings and video lectures, including full mathematics courses.
  • Mathologer - Visual videos on recreational mathematics and selected advanced topics.
  • The Math District - Free guides and videos covering prealgebra, algebra, geometry, trigonometry, statistics, calculus, and technical mathematics.

Questions and Answers

Reference Works

Broad Course Notes

  • Evan Chen - MIT. 2012 ~ 2018. Covers Combinatorics, Number Theory, Honors Algebra, Set Theory, Real Analysis, Graph Theory, and more.
  • Dexter Chua - Harvard. 2013 ~ 2018. Covers Analysis, Probability, Linear Algebra, Complex Analysis, Numerical Analysis, Statistics, Optimization, Algebraic Topology, Quantum Field Theory, and more.

General Mathematics

  • Paul's Online Math Notes - Free notes, examples, and practice problems for algebra, calculus, and differential equations, by Paul Dawkins at Lamar University.
  • UBC CLP Calculus Textbooks - Free textbooks, problem books, and source files for a four-course university sequence in differential, integral, multivariable, and vector calculus.

Foundations and Logic

Logic

Set Theory

Type Theory

Category Theory

Formal Mathematics and Theorem Proving

  • Lean - Open-source interactive theorem prover and programming language based on dependent type theory, used for formal mathematics and software verification.
  • mathlib - Community-maintained mathematical library for Lean 4, with formalized theories, proof tactics, programming infrastructure, and generated documentation.
  • Mathematics in Lean - Free textbook with examples and exercises for mathematicians learning Lean 4 and mathlib.
  • Theorem Proving in Lean 4 - Free official textbook on dependent type theory, propositions, tactics, inductive types, type classes, and constructing verified proofs in Lean 4.
  • Natural Number Game - Interactive Lean 4 game that introduces theorem proving through guided exercises about natural numbers.
  • Isabelle - General-purpose interactive theorem prover with higher-order logic and set-theory environments for formalizing mathematics and computer science.
  • Archive of Formal Proofs - Refereed collection of Isabelle proof libraries, examples, and scientific developments maintained against current Isabelle releases.
  • Rocq Prover - Open-source interactive theorem prover and dependently typed programming language for mechanized mathematics and verified software.
  • Mathematical Components - Rocq libraries and tools for large-scale formalized mathematics, including substantial developments in algebra and analysis.
  • Metamath - Minimal formal language and proof verifier with explicit, inspectable proofs built from simple foundations.
  • Courses Using Lean - Maintained, filterable index of university courses that teach Lean or use it for mathematics, logic, programming, and formal verification.
  • Lean-related Conferences and Events - Maintained calendar and archive of conferences, workshops, and tutorials about Lean, mathlib, and formalized mathematics.

Algebra

  • Elementary Algebra 2e - Free one-semester textbook from OpenStax with web and PDF editions, worked examples, practice exercises, chapter reviews, and answer keys.

Abstract Algebra

Linear Algebra

Group Theory

Ring and Field Theory

Commutative Algebra

  • The CRing Project - Open-source work-in-progress textbook introducing commutative algebra for readers with elementary abstract algebra, with an emphasis on foundations for algebraic geometry.
  • Commutative Algebra Notes - Free graduate course notes by Mel Hochster, from an introductory course through advanced topics such as Cohen-Macaulay rings, multiplicities, and étale maps.

Representation Theory

Number Theory

Algebraic Number Theory

Analytic Number Theory

Combinatorics and Discrete Mathematics

Graph Theory

Geometry and Topology

Differential Geometry

Algebraic Geometry

General Topology

Algebraic Topology

Analysis

Real Analysis

Complex Analysis

Functional Analysis

Operator Theory

  • Lecture Notes on Operator Algebras - Free graduate notes by John M. Erdman on Hilbert-space operators, Banach algebras, spectral theory, C*-algebras, compact operators, and K-theory.

Harmonic Analysis

Measure Theory

Differential Equations and Dynamical Systems

Ordinary Differential Equations

Partial Differential Equations

Ergodic Theory and Chaos

  • Chaos: Classical and Quantum - Maintained technical webbook on dynamical systems, periodic orbits, deterministic chaos, statistical mechanics, and quantum chaos.

Probability and Statistics

Probability

Stochastic Processes

Statistics

Statistical Learning

Algebraic Statistics

Numerical and Computational Mathematics

Numerical Analysis

Optimization, Operations Research, and Control

  • Convex Optimization - A free textbook with slides, exercises, code, and course material by Stephen Boyd and Lieven Vandenberghe.
  • Nonlinear Optimization - Free graduate MIT course covering analytical and computational methods for unconstrained and constrained optimization.
  • CVXPY - Open-source Python modeling language for convex, geometric, quasiconvex, and related optimization problems.

Control Theory

Mathematical Physics

Interdisciplinary Mathematics

Mathematics for Computer Science

Mathematics for Machine Learning

  • Mathematics for Machine Learning - Free textbook on the linear algebra, geometry, calculus, probability, and optimization needed to study machine learning.
  • Deep Learning: Foundations and Concepts - Free-to-read online textbook by Christopher M. Bishop and Hugh Bishop, with a probability-based treatment of modern deep-learning models and methods.
  • Mathematics of Machine Learning - Free graduate MIT course on the mathematical and statistical foundations of machine learning, with lecture notes, assignments, and problem-set solutions.
  • Probabilistic Machine Learning - Introductory and advanced books by Kevin Murphy, with free draft PDFs, code, figures, exercises, and teaching resources.

Information and Coding Theory

  • Information Theory - Free graduate MIT course on entropy, source and channel coding, rate distortion, Gaussian channels, feedback, and multi-user information theory.
  • Essential Coding Theory - Free graduate MIT course on error-correcting codes, Hamming spaces, and the algebraic and complexity aspects of coding theory.
  • Error Correction Zoo - Maintained reference database of classical, quantum, and hybrid error-correcting codes, with definitions, relationships, and references.

Mathematical Finance

Mathematical Biology

Signal Processing

Mathematical Practice

AI for Mathematics

This section covers AI-assisted theorem proving, formalization, and mathematical discovery. Resources that teach the mathematics used in machine learning appear under Mathematics for Machine Learning.

  • LeanDojo v2 - Open-source framework and benchmark data for training, evaluating, and deploying AI-assisted theorem provers for Lean 4.
  • miniF2F - Cross-system benchmark of formalized olympiad, high-school, and undergraduate problems for evaluating automated theorem provers.
  • AlphaGeometry - Open research implementation combining learned guidance with symbolic deduction for olympiad geometry problems.
  • First Proof Project - Independent project that publishes research-level problems, evaluation methods, solutions, and expert commentary for assessing AI systems in mathematics.
  • Math AI Seminar - University of Washington research seminar on formalization, theorem proving, mathematical AI, and machine-learning applications in mathematics, with a multi-year event archive.
  • b=M²L - Barcelona Mathematics and Machine Learning colloquia on the interaction between mathematics and machine learning, with multi-year editions and recorded talks.
  • AI for Mathematics and Theoretical Computer Science - Simons Institute and SLMath workshop archive with recorded talks on proof assistants, automated reasoning, machine learning, and mathematical discovery.
  • Mathematics and AI - University of Washington Fall 2025 graduate course with public slides, Lean examples, projects, and readings on machine learning, formalization, autoformalization, mathematical discovery, and ethics.
  • AI-Assisted Mathematical Discovery - Open thirteen-week course with notes, notebooks, exercises, and case studies on coding agents, machine learning, language models, reinforcement learning, and experimental mathematical research.

Mathematical Software and Tools

  • Symbolab - Online solver with step-by-step solutions, graphing, calculators, and practice tools. Some solution steps and study features require a paid plan.
  • Desmos - Browser-based graphing calculator with geometry, tables, statistics, and classroom activities.
  • Wolfram Alpha - Computational knowledge engine for evaluating expressions, plotting functions, and answering structured queries. Some advanced features require a paid plan.
  • Maxima - Free computer algebra system for symbolic and numerical calculations, plotting, and scripting.
  • SymPy - Python library for symbolic algebra, calculus, equation solving, matrices, and code generation.
  • SageMath - Free mathematics software combining computer algebra, numerical computation, geometry, statistics, and visualization through Python.
  • MathFlow - C# math expression library with symbolic computation (differentiation, simplification, equation solving).
  • GeoGebra - Interactive mathematics suite for geometry, graphing, algebra, spreadsheets, statistics, and calculus.
  • Macaulay2 - Research system for algebraic geometry and commutative algebra with specialized computational packages.
  • Singular - Computer algebra system for polynomial computations in commutative algebra, algebraic geometry, and singularity theory.
  • GNU Octave - Free environment for numerical computing, linear algebra, plotting, and MATLAB-compatible scripts.
  • Magma - Subscription-based computer algebra system for research in algebra, number theory, algebraic geometry, and combinatorics.
  • Maple - Paid computer algebra and numerical mathematics system for symbolic computation, visualization, and programming.
  • MATLAB - Paid numerical computing environment by MathWorks.
  • Wolfram Mathematica - Paid technical-computing system for symbolic and numerical mathematics, visualization, programming, and data analysis.
  • Free Math - Free, open-source software for recording, organizing, and reviewing digital mathematics homework without solving the problems automatically.
  • RunMat - Runtime for MATLAB-syntax array math with automatic CPU/GPU execution.

History and Mathematics Education

  • MacTutor History of Mathematics Archive - University of St Andrews archive of mathematician biographies, histories of mathematical topics, timelines, institutions, and related reference material.
  • NRICH - Free curriculum-linked problems, investigations, articles, and teacher guidance from the University of Cambridge for learners aged 3 to 18.

Community and Further Reading

Journals and Magazines

Blogs

  • BetterExplained - Maintained by Kalid Azad.
  • ILoveMaths - Mathematics lessons and activities for Indian classes 6 to 12.
  • 3Blue1Brown - Animated explanations of mathematics by Grant Sanderson.
  • Math Is Fun - Lightweight lessons and reference pages for school mathematics.

Meetings and Conferences

  • MathsJam - Monthly local recreational mathematics and puzzle meetups, plus an annual gathering in the United Kingdom.
  • Talking Maths in Public - Biennial conference for mathematics communicators in the United Kingdom.
  • Bridges - Annual conference on mathematical connections in art, music, architecture, and culture.
  • Theoretical Computer Science - Curated resources on computation, complexity, logic, algorithms, cryptography, and related mathematics.
  • Scientific Computing - Curated software for numerical analysis, scientific computing, meshing, solvers, and visualization.
  • Optimization - Courses, books, notes, and software across mathematical optimization and operations research.
  • Interactive Mathematics - Tools and libraries for building interactive mathematical explanations and explorations.
  • AI for Mathematics - Research index covering AI-assisted mathematical reasoning, discovery, formal proof, and related datasets. Individual entries require independent review.

2. Official Technical Reference & Guides (rossant/website)

File: README.md

To build with:

  • pelican master
  • pelican-plugins, rossant fork, ipynb branch (to add IPython notebook reader that converts a notebook to Markdown, by keeping or not the input code with keep_input metadata field)
  • need pelican_plugin-render_math
  • Python-Markdown, rossant fork, codehilite-lang-param branch (to set Python as default language for code highliting)

File: content/pages/books.md

Title: Books

I've written two books on Python for data science.

IPython Cookbook

september 2014

text
[](http://ipython-books.github.io/cookbook/)



	<h3>[IPython Interactive Computing and Visualization Cookbook](http://ipython-books.github.io/cookbook/)</h3>
	Packt Publishing (520 pages, 30$)

	
		This advanced-level book covers a wide range of methods for data science with Python:
	

	
		<li>Advanced interactive computing in the IPython notebook</li>
		<li>High-performance computing with Python</li>
		<li>Statistics, machine learning, data mining</li>
		<li>Signal processing and mathematical modeling</li>

IPython Minibook, second edition

october 2015

text
[](http://ipython-books.github.io/minibook/)



    <h3>[Learning IPython for Interactive Computing and Data Visualization, second edition](http://ipython-books.github.io/minibook/)</h3>
    Packt Publishing (175 pages, 25$)

    
        This book is a beginner-level introduction to Python for data analysis:
    

    
        <li>Interactive Python with IPython</li>
        <li>Data analysis with pandas</li>
        <li>Numerical computing with NumPy/SciPy</li>
        <li>Data visualization with matplotlib</li>

File: content/pages/projects.md

Title: Projects

Platform for large-scale electrophysiological data analysis

Next-generation electrophysiological recording devices require scalable data analysis platforms. I'm leading the development of a new Python-based platform that will let scientists analyze their terabytes of data remotely and effectively in the Web browser (thanks to the IPython notebook).

I'm focusing on raw data visualization and extraction of spike trains from raw extracellular recordings (spike sorting).

High-performance big data visualization

I've developed a technology for high-performance interactive visualization of big data based on optimal use of the graphics processing unit (GPU). It is based on modern OpenGL (GLSL shaders) and it uses techniques originally created for 3D video games.

I'm now a core developer of Vispy, a collaborative project in Python that aims at bringing to scientists high-level, user-friendly visualization facilities based on this technology.

I'm also interested in bringing Vispy to the browser. This will let scientists visualize huge volumes of data remotely thanks to WebGL, a browser-based implementation of OpenGL.

Python for data science

I've written two books on Python for data science and interactive computing.

The first book is a beginner-level introduction to Python for data analysis and numerical computing.

The second book is an advanced-level guide to data science with Python, covering technical aspects of interactive and high-performance computing as well as applied mathematics topics such as statistics, machine learning, signal processing, computer vision, numerical optimization, network data analysis, and mathematical modeling for data science. The book contains more than 100 hands-on recipes applied on real-world examples.

Open data

I analyze public datasets with Python and I make data visualizations.

File: content/pages/resume.md

Title: Resume

I am a former student of École Normale Supérieure, where I studied mathematics and computer science. I also have a PhD in neuroinformatics.

Here is a resume of my current and past professional activities.

Professional experience

Analysis of large-scale electrophysiological data

2012–present (postdoc)

I'm developing algorithms and software for analyzing large volumes of electrophysiological data (high-density multielectrode arrays).

Correlations in neural coding

2009–2012 (PhD)

I've developed programs for analyzing intracellular recordings with the help of spiking neuron models. I've also demonstrated theoretically and experimentally a key role for fast temporal correlations in neural coding. My dissertation is available for download here (PDF, 14MB).

Statistical data analysis of retinal spiking data

2008 (six-months research internship)

I've developed a statistical framework for analyzing and modeling the multineuronal spiking activity of retinal ganglion cells recorded with high-density multielectrode arrays (joint work with Kolia Sadeghi). My dissertation is available for download here (PDF, 2MB).

Data analysis of human locomotor trajectories

2007 (two-months research internship)

I've conducted data analysis and modeling investigations of locomotor trajectories recorded with motion-capture cameras.

Publications and conferences

International peer-reviewed scientific journals

Magazines

  • Rossant C (2015, in prep). Visualization of large datasets with VisPy. French Linux Magazine, upcoming Big Data special issue, 2015
  • Rossant C (2015, in prep). Introduction to scikit-learn. French Linux Magazine, upcoming Big Data special issue, 2015
  • Rossant C (2014). VisPy: visualisation interactive haute performance. French Linux Magazine, Scientific Python special issue, summer 2014

Books

Conferences

  • Kadir S, Rossant C, Goodman D, Schulman J, Hunter M, Belluscio M, Buzsaki G, Harris D (2014). Spike sorting for large dense arrays. Society for Neuroscience Annual Meeting, Washington, USA (poster)
  • Klein A, Rougier N, Rossant C, Larson E, Campagnola L (2014). Introducing VisPy's high level modules: easy yet powerful visualization for everyone. EuroSciPy, Cambridge, UK (talk by Klein A)
  • Rossant C, Harris K (2013). Spike sorting for large dense electrode arrays: User interface software. Society for Neuroscience Annual Meeting, San Diego, USA (poster)
  • Rossant C (2013). Spike-based computation in cortical networks: theory and data, Mathematical Neuroscience Laboratory, CIRB, Collège de France, Paris
  • Campagnola L, Klein A, Rossant C, Rougier N (2013). A Modern and Interactive Visualization Framework, EuroSciPy, Brussels, Belgium (talk by Rougier N)
  • Rossant C, Harris K (2013). High-performance interactive data visualization in Python. EuroSciPy, Brussels, Belgium (poster)
  • Rossant C, Harris K (2013). Spike sorting for large dense electrode arrays: User interface software. Organization for Computational Neuroscience Meeting, Paris (poster)
  • Rossant C, Harris K (2013). Spike sorting for large dense electrode arrays: User interface software. UCL Neuroscience Symposium, London (poster)
  • Rossant C (2013). Semi-automatic algorithms for manual spike sorting, Cortical Processing Laboratory Seminar, UCL, London
  • Rossant C, Harris K (2013). Spike sorting for large dense electrode arrays: User interface software. British Neuroscience Association's biennial meeting, London (poster)
  • Rossant C (2011). Coincidence detection in noisy neurons, Laboratory of Computational Neuroscience Seminar, EPFL, Lausanne, Switzerland
  • Rossant C, Leijon S, Magnusson AK, Brette R (2011). Sensitivity of noisy neurons to coincident inputs. Workshop on Mean-field methods and multiscale analysis of neuronal populations, CIRM, Marseille, France (poster)
  • Brette R, Rossant C, Benichoux V (2011). What's new in Brian? Python in Neuroscience Euroscipy Workshop, Paris (talk)
  • Rossant C, Brette R (2010). Coincidence detection in noisy neurons. Society for Neuroscience Annual Meeting, San Diego, USA (poster)
  • Rossant C, Brette R (2010). Coincidence detection in noisy neurons. NeuroComp, Lyon, France (poster)
  • Rossant C, Fontaine B, Goodman DF (2010). Playdoh: a lightweight Python library for distributed computing and optimisation. Euroscipy 2010, Paris (talk)
  • Rossant C, Brette R (2010). Coincidence detection in active neurons. Cosyne, Salt Lake City, USA (poster)

Workshops and tutorials

  • Rossant C, Rougier N, Klein A, Campagnola L, Larson E, Zaid I, Hunter M (2015). VisPy sprint, Collège de France, Paris (organization and participation to the sprint)
  • Rossant C, Harris K (2014). File formats for large-scale electrophysiology. Janelia Workshop, Janelia Farm Research Campus, Ashburn, USA (invited talk)
  • Rossant C (2014). The new interactive features of the IPython 2 notebook, EuroSciPy, Cambridge, UK (advanced tutorial)
  • Rougier N, Klein A, Rossant C (2014). VisPy sprint, EuroSciPy, Cambridge, UK (sprint)
  • Rossant C, Rougier N (2014). Scientific Data Visualization, LoOPS network meeting, Orsay University. Presentation and tutorials on HDF5 and VisPy
  • Rossant C, Klein A, Campagnola L, Rougier N (2014). Organization of the VisPy code camp at the European Synchrotron Radiation Facility (ESRF) in Grenoble
  • Rougier N, Klein A, Rossant C (2013). Python visualization, EuroSciPy, Brussels, Belgium (sprint)
  • Brette R, Stimberg M, Benichoux V, Rossant C, Goodman D, Fontaine B (2013). Advanced modelling of spiking neural networks with Brian. Organization for Computational Neuroscience Meeting, Paris (tutorial)
  • Rossant C, Harris K (2013). Next-generation software for spike sorting with large multi-electrode probes. Janelia Spike Sorting Workshop, Janelia Farm Research Campus, Ashburn, USA (invited talk)

Grants and awards

  • Research grant (2014-2015). Cooperation between Paris-Sciences-Lettres (PSL) and University College London.

Education and diplomas

  • 2012: PhD in computational neuroscience at the ENS and UPMC with highest honors
  • 2008: Master's Degree in computer science (MPRI) at the ENS with highest honors
  • 2005: Bachelor's Degree in mathematics (FIMFA) at the ENS with highest honors
  • 2005: Entry to the École Normale Supérieure (ENS) through competitive examinations in maths and physics
  • 2002: Preparatory classes (MPSI/MP*) at Lycée Masséna, Nice
  • 2002: High school diploma (A-level equivalent) with highest honors

Teaching

  • 2009-2012: Teacher assistant in mathematics and computer science at ENS and UPMC (Bachelor and Master levels): probability theory, measure theory, statistics, calculus, real analysis, dynamical systems, signal processing, scientific Python

  • 2009-: Private lessons in mathematics to students in high school and preparatory classes

  • 2008-2009: Voluntary teaching to high school students at TALENS association

Technical skills

  • Applied mathematics: statistics, data mining, machine learning, signal processing, image processing and computer vision, mathematical optimization, dynamical systems, graph theory, high-dimensional data analysis, computational geometry
  • Scientific Python, MATLAB, data visualization
  • Programming on graphics cards: CUDA, OpenCL
  • Computer graphics: OpenGL, DirectX, ray tracing, video game development, Photoshop, Illustrator
  • Website development: HTML5, CSS3, JavaScript, Backbone, D3js, PHP/MySQL
  • Mobile and windows applications: C#, .NET, Windows Phone, Qt
  • Other languages: C, C++, assembly, Perl, Java

File: content/2012-07-02-python-ide-windows.md

Title: Build a Python IDE for Windows with Notepad++ and IPython
Tags: python

I've been looking for a good and free Python IDE for Windows for a long time. I first used
Eclipse and Pydev, but
Eclipse is a bit of a "usine à gaz". I tried
Spyder but it crashed too often
and it was not flexible enough. I ended up using
Notepad++, a really fast and powerful
text editor, with the great command-line interpreter
IPython. Notepad++ is extremely light to use,
opens within a second, and contains most important features a Python
developer wants (syntax highlighting, automatic indentation, code
folding...). However, it does not include a native way for running
Python scripts. On the other hand, IPython is a widely used and
extremely powerful Python interpreter that is well adapted for
scientific computing. It allows to run scripts in a command-line
interface, and offers the possibility to continue an interactive session
afterwards. It also includes a debugger and many more features.

Unfortunately, using both Notepad++ and IPython in a convenient way
during an interactive session, where one edits a script and runs it many
times, is not straightforward. One can simply open both programs, with
IPython opened in the script directory, and call the magic %run
script.py command repeatedly. However, when calling %run several
times, the Python script is correctly reloaded every time, but not the
imported modules that could also have been edited in Notepad++. In other
words, if one runs a script which depends on different modules, and
edits one such module, those modifications won't be effective at the
next %run command. One has to import and reload that specific module,
which can be tiresome when editing a lot of different modules. There
appears to be no way of resetting the whole IPython environment and
reloading all modules. So typically one has to close and open again
IPython in the right working directory. This is cumbersome since every
edition-evaluation loop requires to close the IPython interpreter, open
it again, put the window at a convenient place on the screen, go in the
right directory, and run the script. Over and over again, even when
changing just one line of code in an imported script.

Those little issues might seem unimportant, but they can really hurt
productivity and prevent the user from focusing on its core work.
Therefore I've been looking for a way of automating this interactive
edition-evaluation loop with Notepad++ and IPython. The goal is to let
the user press a single keyboard button (e.g. F6) in a Python script
opened in Notepad++ in order to execute it in IPython. One should have
also a simple way of reloading all modules if needed. Here is what I
came up with. This method may certainly be improved, and it could even
be adapted to other OS than Windows.

This method works with Python 2.7, IPython 0.13 and Notepad++ 6.1.3 (it
will probably work with other versions, but it may require small
modifications).

First, create a custom IPython script that will define two new magic
commands in IPython called %cdrun and %cdrunkill. Both open a new
IPython interpreter, set a specific working directory, and run a Python
script. The second command also kills any existing IPython interpreter
(corresponding to a hard restart of IPython). This script will be loaded
at every IPython launch, so it should be placed here:

text
:::text
C:\Users\\.ipython\profile_default\startup\cdrun.py

This file contains the following code.

text
:::python
import os
from subprocess import Popen, PIPE

ip = get_ipython()

def kill_python():
    # kill all other python processes
    pid = str(os.getpid())
    cmd = ['tasklist', "/FO", "LIST", "/FI", "IMAGENAME eq python.exe"]
    r = Popen(cmd, stdout=PIPE).communicate()[0]
    r = r.split("\n")
    ps = []
    for line in r:
        if line[:3] == 'PID':
            ps.append(line[4:].strip())
    if pid in ps:
        ps.remove(pid)
    n = len(ps)
    if n == 0:
        return
    args = " ".join(["/pid %s" % p for p in ps])
    cmd = "taskkill /F %s" % args
    os.system(cmd)
    return n

def cdrun(self, arg):
    h, t = os.path.split(arg)
    ip.magic("cd %s" % h)
    ip.magic("run %s" % t)

def cdrunkill(self, arg):
    kill_python()
    h, t = os.path.split(arg)
    ip.magic("cd %s" % h)
    ip.magic("run %s" % t)

ip.define_magic('cdrun', cdrun)
ip.define_magic('cdrunkill', cdrunkill)

The kill_python function kills all Python processes except the
current one. Hence this script allows to automatically close a previous
IPython interpreter and open a new one, resolving the module reloading
issue. If you want to open a new IPython interpreter without killing all
Python processes (for example, with a multicore computer, launching
several scripts in different IPython interpreters can make them run in
parallel on several CPUs), you can use the first command cdrun.

Next, you can specify the screen location and the size of the Ipython
window. It is very convenient because it allows you to put the
Notepad++, say, on the left of the screen, and the IPython interpreter
on the right, so that you don't have to manually move the IPython window
every time you launch a new interpreter (otherwise Windows tends to put
it at random locations). To do this, edit the window properties of the
IPython prompt (see method on
this website).

Third, define a new macro in Notepad++, by editing or creating the
following file:

text
:::bash
# in C:\Users\\AppData\Roaming\Notepad++\shortcuts.xml
ipython -i -c "%cdrun $(FULL_CURRENT_PATH)"  
ipython -i -c "%cdrunkill $(FULL_CURRENT_PATH)"

The buttons #117 and #118 correspond to F6 and F7 here. If other
commands exist, just add the <Command> lines in the
** section. If those shortcuts don't work, it may
be because they are already assigned to other commands: go in Settings,
then Shortcuts, and remove them.

Also, do not edit this XML file with Notepad++, because your
modifications may be discarded when you close Notepad++. Instead, ensure
that Notepad++ is closed, edit the file with Notepad, and reopen
Notepad++.

Now, when you open a Python script in Notepad++, you can press F6. It
will launch an IPython window and execute the script. At the end of the
execution, you can interact in the IPython interpreter. If you need to
execute the script again, just use the following command:

text
:::python
%run script.py

This will reload script.py, but not other modules. If you need to
reload everything, press F7 in Notepad++: it will close the IPython
window and open a new one, at the same location on the screen. This
gives you the illusion of a very light and basic Python IDE, and allows
you to benefit from both the great editing features of Notepad++ and the
powerful IPython interpreter at the same time.

Note: you can automate the process of re-evaluating the script in
IPython. Here, you need to: save the script in Notepad++, set the focus
to IPython (with the mouse or with ALT+TAB), and type again the %run
command (or selecting it in the command history). With the very powerful
tool AutoHotkey, you can create a simple
script for automating these keystrokes. Create a text file named
"ipython_update.ahk" somewhere and put the following code:

text
:::ahk
SetTitleMatchMode, 2
IfWinExist Notepad`+`+
{
    WinActivate
    Send ^s
    SetTitleMatchMode, 1
    IfWinExist ipython
    {
        WinActivate
        Send `%run %1%`r
    }
}

Now, in Notepad++, create a new command for launching this script with
the current file as parameter:

text
:::bash
autohotkey [YOURPATH]\ipython_update.ahk $(FILE_NAME)

Ensure that the autohotkey.exe binary is in the Windows Path. Now,
pressing F8 in Notepad++ will automatically set the focus to IPython and
run your script.

Note for Linux users: Here is a Python function for killing all
Python processes except the current one on Linux that might be useful.

text
:::python
def kill_python():
    """
    Linux version (not very well tested, might need some tweaking)
    """
    pid = str(os.getpid())
    cmd = "ps -ef | grep python | awk '{print $2}'"
    r = commands.getoutput(cmd)
    r = r.split('\n')
    if pid in r:
        r.remove(pid)
    n = len(r)
    r = ' '.join(r)
    cmd = 'kill %s' % r
    os.system(cmd)
    return n

File: content/2012-07-04-introduction-to-bayesian-thinking.md

Title: Introduction to Bayesian thinking
Tags: maths

An updated version of this post can be found in the IPython Cookbook. It will also be available here soon.

File: content/2012-07-12-prenoms-et-reussite-au-bac.md

Title: Prénoms et réussite au bac
Tags: opendata

Le sociologue Baptiste Coulmont, auteur de
Sociologie des prénoms, s'est
intéressé aux corrélations entre le prénom des candidats au baccalauréat
2012 et leur réussite à cet examen (voir ici pour l'étude originale sur
le blog de
l'auteur).
Ce travail a été largement relayé par les
médias,
car les résultats sont particulièrement intéressants. Il s'avère que
certains prénoms réussissent mieux que d'autres: les Madeleine et autres
Côme réussissent bien mieux (plus de 25% de mentions TB chez ces
candidats) que les Youssef ou Nabil (aucune mention TB cette année). Ce
n'est probablement pas le prénom en lui-même qui influe sur le résultat,
mais plutôt le fait que le prénom est souvent le reflet de certains
critères sociologiques (classe sociale, origine géographique, profession
des parents...). Ce sont ces derniers critères qui ont une influence
certaine sur les résultats scolaires, démontrant des inégalités
profondes dans le système éducatif actuel.

EDIT: maintenant, vous pouvez connaître les résultats des candidats
portant votre prénom!

J'ai voulu effectuer quelques analyses complémentaires sur ces mêmes
données, en m'intéressant plus aux aspects mathématiques que
sociologiques. Il peut s'agir d'un exemple intéressant dans
l'enseignement d'un cours de probabilité et statistiques, illustrant
notamment les notions de
conditionnement,
d'espérance
conditionnelle,
de raisonnement
bayésien,
ou de divergence de
Kullback-Leibler.

Résultat moyen en fonction du prénom

Nous nous intéressons au profil de résultats: il s'agit d'une courbe
donnant la proportion de candidats pour chaque résultat possible: oral
(O), passable (P), mention assez bien (AB), mention bien (B), mention
très bien (TB). Cela s'appelle aussi une distribution de probabilité sur
l'ensemble des résultats possibles. Voici le profil moyen, pour tous les
candidats et indépendamment du prénom. Il y a environ 17% de candidats
devant passer l'oral, 35% de P, 27% de AB, 14% de B, 7% de TB.
Rappelons que ces chiffres comportent deux biais: il ne s'agit que des
candidats ayant obtenu l'examen après le premier écrit, ou ayant obtenu
la possibilité de passer l'oral. Les candidats ayant échoué après le
premier écrit ne sont pas inclus dans les données. Le deuxième biais
concerne le fait que seuls les candidats ayant accepté de diffuser
publiquement leurs résultats sont inclus.

Maintenant, nous nous intéressants à l'influence du prénom sur le
résultat au bac. Nous allons utiliser deux approches mathématiques: la
première utilise l'espérance conditionnelle, la seconde utilise la
divergence de Kullback-Leibler. Dans les deux cas, l'idée est de
comparer le profil moyen avec le profil pour chaque prénom. Cela revient
à étudier la distribution de probabilité conditionnelle à la donnée du
prénom. En d'autres termes, si un candidat pris au hasard a 7% d'avoir
eu mention TB, le fait de savoir qu'il s'appelle Côme, Nicolas, ou
Youssef change-t-il cette probabilité? Cette question correspond au
principe mathématique de conditionnement. Du point de vue de
l'espérance conditionnelle, on peut se demander, si la moyenne générale
au bac est de 12 chez tous les candidats, combien de points en plus on
peut attendre chez un candidat en connaissant son prénom? D'un point de
vue bayésien, on peut aussi voir la question ainsi: connaissant le
profil de résultat moyen (prior), le fait de connaître le prénom
change-t-il le profil de résultat (posterior)?

Moyenne au bac

Dans cette première approche, nous nous intéressons à la moyenne des
candidats, sachant ou non leurs prénoms. Mathématiquement, il s'agit
respectivement de l'espérance de la distribution moyenne du résultat, et
de l'espérance conditionnelle de cette distribution sachant le
prénom du candidat. Pour cela, nous assignons d'abord une note moyenne à
chaque mention: 9 pour O, 11 pour P, 13 pour AB, 15 pour B, 17 pour TB.
Il ne s'agit bien sûr que d'une approximation, puisque nous n'avons
accès qu'à la mention et pas à la note moyenne de chaque candidat. Chez
tous les candidats (ayant réussi ou devant passer l'oral, donc), la
moyenne globale est de 12.2.

Nous pouvons ensuite, pour chaque prénom, calculer l'écart, par rapport
à cette moyenne, de la moyenne des candidats portant ce prénom (ou
espérance conditionnelle). Nous trouvons que cet écart varie entre -2.1
(Nahida) et +2.1 (Gaëtane), soit une bonne mention d'écart. Autrement
dit, une Nahida a en moyenne une mention en-dessous de la mention
moyenne des candidats, comparé à une mention au-dessus pour une
Gaëtane. Voici le profil de résultat des six prénoms les plus
en-dessous de la moyenne:

Voici les résultats complets pour les 30 prénoms les plus en-dessous,
avec l'écart à la moyenne.

text
:::text
Nahida       -2,07
Alyson       -1,97
Imad         -1,91
Nadir        -1,80
Mariama      -1,75
Bilel        -1,65
Fatma        -1,63
Abdelkader   -1,60
Manuella     -1,59
Aziza        -1,57
Yassin       -1,57
Mounir       -1,57
Moussa       -1,56
Siham        -1,55
Patrice      -1,53
Kamel        -1,53
Giovanni     -1,53
Dalila       -1,47
Abdel        -1,45
Erwin        -1,42
Nordine      -1,41
Khalid       -1,40
Latifa       -1,40
Youssef      -1,38
Sofian       -1,38
Souad        -1,37
Soumia       -1,37
Bilal        -1,36
Alysson      -1,36
Lamia        -1,35

Du côté des prénoms avec l'écart le plus au-dessus de la moyenne, voici
les profils de résultats.

text
:::text
Gaetane        2,13
Madeleine      1,94
Ariane         1,82
Come           1,76
Marguerite     1,73
Isaure         1,71
Mahaut         1,61
Marie-Anne     1,58
Hannah         1,51
Melisande      1,46
Viviane        1,45
Quitterie      1,43
Marie-Astrid   1,43
Guillemette    1,38
Anouk          1,35
Irene          1,34
Blaise         1,33
Arielle        1,33
Solange        1,33
Leon           1,31
Domitille      1,29
Celeste        1,23
Noemi          1,23
Cleo           1,23
Hortense       1,22
Baudouin       1,19
Jack           1,19
Iris           1,17
Jolan          1,17
Gaspard        1,16

Destin des prénoms

Avec la seconde méthode, nous nous intéressons à la quantité
d'information obtenue avec la connaissance du prénom. Autrement dit,
posons-nous d'abord la question suivante:

  • Quel résultat attendre d'un candidat pris au hasard?
  • Réponse: voir le profil de résultat moyen au début de cet article,
    supposé connu a priori (7% de mention TB, 14% de mentions B, etc.).

Ensuite:

  • Quel résultat attendre d'un candidat pris au hasard, si l'on nous
    donne son prénom    ? Plus précisément, à quel point notre attente du
    résultat est-elle modifiée par la connaissance de son prénom    ?
  • Réponse: la divergence de Kullback-Leibler entre la
    distribution moyenne, et la distribution conditionnée au prénom,
    donne une réponse quantitative à cette question.

Cette divergence constitue une sorte de métrique entre deux
distributions. Une divergence quasi-nulle signifie que les distributions
sont quasi-identiques, donc que notre attente du résultat n'est pas
modifiée par la connaissance du prénom. A l'inverse, une divergence plus
élevée signifie que la connaissance du prénom change de manière
importante le résultat que l'on peut attendre du candidat (voir ici
pour des
détails).
Les graphiques ci-dessous illustrent cela.

Voici d'abord les six prénoms associés à la divergence la plus élevée.
Nous reconnaissons les prénoms obtenus plus haut, sauf que les prénoms
réussissant le mieux sont mélangés avec ceux réussissant le moins. En
effet, cette mesure signifie à quel point les profils de résultats sont
différents de la moyenne, mais pas la direction de cette différence, qui
peut être meilleure (plus de bonnes mentions) ou moins bonne (plus de
mauvais résultats).

Voici les 30 prénoms associés à la divergence la plus élevée.

text
:::text
Nahida
Alyson
Imad
Nadir
Alysson
Gaetane
Tina
Mariama
Patrice
Madeleine
Aziza
Come
Fatih
Bilel
Manuella
Moussa
Fatma
Giovanni
Abdelkader
Cleo
Elia
Erwin
Isaure
Abdel
Siham
Zelie
Yassin
Mounir
Juliane
Leon

Maintenant, nous pouvons aussi nous intéresser aux prénoms avec la
divergence la plus faible, autrement dit les prénoms les plus
représentatifs de l'ensemble des candidats. Ces prénoms sont
probablement répartis de manière uniforme parmi tous les candidats, sans
être associés, par exemple, à une classe sociale particulière qui aurait
des résultats très différents de la moyenne. Ainsi, un Valentin ou une
Emilie a un profil de résultat tout à fait comparable à la moyenne
globale.

text
:::text
Valentin
Emilie
Nicolas
Alexandra
Clement
Laura
Thomas
Arnaud
Remy
Alexandre
Helene
Hugo
Mario
Bastien
Sebastien
Estelle
Xavier
Romain
Quentin
Eva
Guillaume
Rayan
Vincent
Adrien
Sabine
Ines
Lucas
Sarah
Floriane
Virgile

En conclusion, nous avons donné des méthodes pour trouver les prénoms
qui influent le plus (positivement ou négativement) sur le résultat au
bac, et à l'inverse pour trouver ceux qui ont le moins d'impact sur le
résultat et qui sont le plus proches possible des résultats globaux. Il
y a probablement beaucoup d'autres choses à analyser dans ces données.

Méthodes

Données. Les données concernent les résultats de 346851 candidats,
toutes séries confondues (générales et technologiques), qui ont au moins
eu l'oral et qui ont accepté de rendre disponible publiquement leur
résultat. Tous ces candidats portent 18473 prénoms différents. Pour les
analyses, nous n'avons considéré que les prénoms portés par au moins 20
candidats, soit 1211 prénoms différents représentant 310145 candidats.

Modélisation. L'ensemble des candidats est noté
$\Omega$, l'ensemble des 1211 prénoms est noté
$E$, l'ensemble des résultats possibles est noté
$F$. L'ensemble $\Omega$ est muni de la tribu
discrète et de la mesure de probabilité uniforme. Les variables
aléatoires $X(\omega)$ et $Y(\omega)$ sont
des variables aléatoires à valeurs dans $E$ et
$F$, respectivement, et donnant le prénom et le résultat de
chaque candidat. Voici les formules utilisées.

  • La probabilité de $y$ est:

    $$
    p[y]=\frac{\textrm{card}\left{\omega \in
    \Omega \mid Y(\omega)=y\right}}{\textrm{card}(\Omega)}
    $$

  • La probabilité conditionnelle de $y$ sachant
    $x$ est:

    $$
    p[y\mid x] = \frac{\textrm{card}\left{\omega
    \in \Omega \mid X(\omega)=x,
    Y(\omega)=y\right}}{\textrm{card}\left{\omega \in \Omega
    \mid X(\omega)=x\right}}
    $$

  • L'espérance de $y$ est:

    $$
    E[y]=\sum_{y \in F} y \times p[y]
    $$

  • L'espérance conditionnelle de $y$ sachant
    $x$ est:

    $$
    E[y \mid x]=\sum_{y \in F} y \times p[y \mid
    x]
    $$

  • La divergence de Kullback-Leibler entre la distribution
    $p[y]$ et la distribution conditionnelle
    $p[y\mid x]$ est:

    $$
    D(x)=\sum_{y \in F} p[y \mid x] \times
    \log\left(\frac{p[y \mid x]}{p[y]}\right)
    $$

File: content/2012-07-14-frequence-des-prenoms-des-candidats-au-bac-2012.md

Title: Fréquence des prénoms des candidats au bac 2012
Tags: opendata

Je poursuis mon analyse des prénoms des candidats au bac cette année (voir
billet précédent) en
m'intéressant cette fois à la fréquence des différents prénoms portés
par les candidats, indépendamment de leurs résultats. Les 346 581
candidats portent 18 473 prénoms différents. Le prénom le plus porté
est Camille avec 4 848 candidats. Voici les quelques prénoms les plus
utilisés, représentatifs des prénoms à la mode aux alentours de l'année
1994 (nuage réalisé sur Wordle).

text
:::text
Camille     4848
Marie       4354
Thomas      4134
Manon       4131
Nicolas     3851
Alexandre   3831
Maxime      3720
Laura       3702
Marine      3639
Pauline     3630
Marion      3433
Antoine     3370
Julie       3157
Mathilde    3045
Quentin     3036
Guillaume   2913
Pierre      2872
Julien      2854
Kevin       2835
Lea         2780
Anais       2775
Romain      2621
Clement     2615
Melanie     2576
Justine     2564
Sarah       2398
Florian     2298
Chloe       2256
Charlotte   2172
Valentin    2150

Une observation intéressante est que les prénoms rares sont,
paradoxalement, très nombreux. Ainsi, 12 129 prénoms soit 65% de tous
les prénoms ne sont portés que par un seul candidat! Mais ces
candidats ne représentent que 3.5% du total des candidats. Un nombre
important de parents souhaite ainsi donner un prénom unique à leur
enfant, souvent d'ailleurs en trouvant des variantes orthographiques
originales de prénoms plus classiques. Par exemple, le prénom Zinedine
admet comme variantes, entre
autres: Zin-Eddine, Zine-Din, Zine-Dine, Zine-Eddine, Zineddine
ou Zinnedine. D'ailleurs, les prénoms composés représentent 15% de
tous les prénoms (et 2% de tous les candidats).

Poursuivant le raisonnement, on peut s'intéresser au nombre de prénoms
portés par 2, 3, 4 candidats ou plus. Cela revient à étudier la
distribution des fréquences des prénoms. On obtient alors la courbe
suivante.

A gauche est représenté la fraction de prénoms qui sont portés par 1, 2,
... jusqu'à 20 candidats (points bleus). Si 65% des prénoms sont portés
par un seul candidat, 10% le sont par 2 candidats, 5% le sont par 3,
etc. Cette courbe démarre à une valeur importante mais diminue ensuite
extrêmement rapidement. Il s'agit d'une propriété caractéristique de
nombreux phénomènes et qui porte le nom de "longue
traîne".
L'utilisation des mots dans une langue donnée, la répartition des
richesses, la population dans les villes constituent quelques exemples.
Il y a ainsi très peu de villes extrêmement peuplées, et énormément de
villes très peu peuplées, de telle sorte qu'environ 20% de toutes les
villes représentent 80% de la population. Ou encore, 20% d'une
population concentre 80% des richesses. C'est la loi des 80-20, ou
principe de Pareto,
qui quantifie en quelque sorte certaines inégalités.

Ici, ce principe est approximativement vérifié qualitativement mais
non quantitativement: il est en fait plus marqué que la règle des
80-20! Ainsi 80% des candidats sont représentés non pas par 20% des
prénoms, mais par 2.5%! Au delà de l'observation d'un grand nombre de
prénoms rares, on peut faire celle d'un petit nombre de prénoms très
répandus. Ainsi, seuls 0.5% des prénoms servent à nommer la moitié des
candidats. De plus, 10% des prénoms représentent 90% des candidats,
formant ainsi plutôt une règle des 90-10.

Mathématiquement, on peut constater que la distribution des fréquences
des prénoms semble suivre de manière assez approximative une loi de
Zipf, typique d'une
distribution à longue traîne et satisfaisant à une règle de type 80-20.
Cette loi signifie que le nombre de prénoms donné à $n$
candidats est proportionnel à $1/n^s$, avec
$s>1$ un paramètre réel (loi de puissance). On constate en
effet que la distribution des fréquences des prénoms dans un graphique
log-log est
approximativement linéaire, suggérant une loi de puissance
(voir panneau de droite dans la figure précédente). On peut tenter de
faire correspondre une loi de Zipf à ces données, à l'aide d'une méthode
d'estimation statistique. En utilisant la méthode du maximum de
vraisemblance,
on trouve une valeur $s \simeq 1.9$ pour le paramètre.
Cette distribution adaptée aux données est représentée en rouge dans la
figure précédente.

Une autre manière de représenter cette distribution est d'afficher le
pourcentage de candidats couverts par 1, 2, 3... prénoms, en triant les
prénoms par ordre décroissant de fréquence. On obtient le graphique
suivant (à droite, échelle log-log) :

Le "coude" observé à gauche est typique d'une loi de Zipf. De gauche à
droite, les prénoms sont classés par fréquence décroissante (le prénom
le plus donné d'abord, Camille, puis le second, Marie, etc.). Au fur et
à mesure que l'on ajoute ces prénoms, la fraction de candidats
représentés augmente très rapidement (partie gauche du coude). Puis
viennent les prénoms beaucoup plus rares, portés par un nombre très
faible de candidats (partie droite). La courbe croît alors très
lentement, le nombre de candidats couverts n'augmentant que d'une
fraction infinitésimale à chaque prénom.

La répartition des prénoms est ainsi particulièrement hétérogène, avec
d'un côté des prénoms très répandus, et de l'autre côté des prénoms
totalement originaux, nouveaux, ou uniques. Y aurait-il deux grandes
tendances chez les parents, entre les "traditionalistes" qui
souhaiteraient utiliser des prénoms classiques (ou des prénoms
d'ascendants familiaux), et les "modernistes" qui souhaiteraient rompre
avec le passé et marquer leur originalité? Notons qu'il est possible de
concilier les deux tendances, en quelque sorte, en utilisant un prénom
du passé en second prénom, comme pour manifester un lien avec les
générations
précédentes.
Par ailleurs, cette tendance observée dans une population très
spécifique (une partie des candidats au bac 2012) est-elle la même que
celle observée plus généralement dans toute la population? Je laisse le
soin aux sociologues d'y répondre.

File: content/2012-08-19-a-mathematical-proof-that-our-world-is-in-three-dimensions.md

Title: A mathematical proof that our world is in three dimensions
Tags: maths

Everyone knows that our world is three-dimensional. It seems indeed obvious that there are three independent spatial directions, no less, no more. But can we really be sure? After all, everything we know about our world comes from our senses and from what our brains interprete. Does the fact that we see in 3D really proves that the world is in 3D? Can we have a mathematical proof, based on experimental observations, that the world is three-dimensional? This question is totally relevant, and several scientists and philosophers, like Immanuel Kant, have been interested in it (see for example this blog post in French for a nice review).

Here, I detail Kant's idea of a rigourous proof that we live in a 3D world. There are three steps:

  1. We model the world as an N-dimensional Euclidean space and we study a mathematical object called a vector field that verifies some natural properties.
  2. We find experimental observations that allows us to infer properties about this object.
  3. We link these properties to the number of spatial dimensions and we conclude that $N=3$.

The physical notions behind this proof are about conservative forces and potential theory. The mathematical notions concern the Laplace operator and harmonic functions.

First, here is an intuitive explanation of this proof (quoting Wikipedia):

The inverse-square law generally applies when some force, energy, or other conserved quantity is radiated outward radially in three-dimensional space from a point source. Since the surface area of a sphere (which is $4\pi r^2$) is proportional to the square of the radius, as the emitted radiation gets farther from the source, it is spread out over an area that is increasing in proportion to the square of the distance from the source. Hence, the intensity of radiation passing through any unit area (directly facing the point source) is inversely proportional to the square of the distance from the point source.

  1. Mathematical model

We model the world as an open set $U$ of $\mathbb R^N$, where $N$ is the number of dimensions. We consider a vector field $F$, that is, a function $F: U \longrightarrow \mathbb R^N$ that maps any point $p$ in $U$ to a $N$-dimensional vector attached to that point. This vector field can represent a force field, like a gravitational field generated by a massive object, or an electrostatic field generated by a charged body. It can also represent a flux, like in transport phenomena: heat transfer, mass transfer or fluid dynamics.

We will assume three natural hypotheses: that this vector field is incompressible (or solenoidal), conservative (or irrotational), and isotropic. One also uses the terminology of Laplacian vector field when the first two hypotheses are satisfied. These hypotheses correspond to a conserved quantity that is radiated outward radially and isotropically from a point source.

Incompressibility occurs when the divergence of the vector field is zero: $\nabla \cdot F=0$. It means that there is no local source or sink, no local creation or annihilation. If the vector field represents a mass transfer, incompressibility means that there is conservation of mass: mass cannot be created or annihilated at any point in space. For a gravitational field, the divergence is zero outside mass sources. Similarly, for an electrostatic field, the divergence is zero outside charge sources.

Irrotationality occurs when the curl of the vector field is zero: $\nabla \times F=0$. It means that the field does not spin locally (in fluid dynamics terms, there is no vorticity). An equivalent property is that the field is conservative, so that the line integral from one point to another is independent of the choice of path connecting the two points (in fact, one has to assume the simple connectedness of $U$ for the equivalence). A conservative field is the gradient of a scalar field: there exists a scalar field $u : U \longrightarrow \mathbb R$ such that $F = \nabla u$. The gravitational force and the electrostatic force are examples of conservative forces. The gravitational vector field is the gradient of the gravitational potential field, and similarly for the electrostatic field.

Finally, isotropy means uniformity in spatial directions, that is, we assume that the vector field $F$ is rotational invariant. For example, the gravitational or electrostatic field generated by a perfectly spherical source is rotational invariant.

Under the two last hypotheses, we can write $F=\nabla u(r)$ where $r$ is the distance to the origin: $r^2 = \sum x_i^2$. The vector field $F$ is the gradient of a scalar field $u$ that depends only on $r$ since it is isotropic. The first hypothesis, $\nabla \cdot F=0$ (outside field sources), implies that $\nabla \cdot \nabla u=0$, that is, $\Delta u=0$ where $\Delta=\sum \frac{\partial^2}{\partial x_i^2}$ is the Laplacian.

The equation $\Delta u=0$ is a partial differential equation (PDE) called the Laplace equation. A function with a null Laplacian is called an harmonic function, so we're looking for an harmonic function $u(r)$.

  1. Experimental observations

It is known since Newton and Coulomb that the laws of gravitation and electrostatics follow inverse-square laws, where the strength of interaction forces is proportional to the inverse square of the distance. This observation has been validated by a number of experiments during the last centuries. The same inverse-square law is observed for light, sound and radiation phenomena.

We will now show that if a vector field satisfying the three preceding properties (like the gravitation law) verifies the inverse-square law, then it implies necessarily that $N=3$, i.e. the world has three dimensions.

  1. Proof

The proof is straightforward once one knows the following expression for the Laplacian in N dimensions and in spherical coordinates:

$$\Delta u(r) = \frac{1}{r^{N-1}} \times \frac{\partial}{\partial r} \left( r^{n-1} \frac{\partial u}{\partial r} \right) + \frac{1}{r^2} \times \Delta_{S^{N-1}} u$$

where $\Delta_{S^{N-1}}$ is the Laplace-Beltrami operator on the (N-1)-hypersphere (an (N-1)-dimensional Riemannian manifold), also called spherical Laplacian.

Since $u$ is isotropic and depends only on $r$, it is constant on hyperspheres (defined by $r=\textrm{constant}$), so that the restriction of $u$ on $S^{N-1}$ is constant, its gradient is null, and $\Delta_{S^{N-1}} u=0$. Therefore:

$$\frac{1}{r^{N-1}} \times \frac{\partial}{\partial r} \left( r^{n-1} \frac{\partial u}{\partial r} \right) = 0$$

hence:

$$\left( r^{n-1} \frac{\partial u}{\partial r} \right) = \textrm{constant}$$

and finally we find that:

$$u(r) = \frac{C}{r^{N-2}}.$$

We omit the integration constant since a potential field is defined up to an additive constant. Hence, the intensity of the vector field $\nabla u$ is proportional to $1/r^{N-1}$. For an inverse-square law, it means that $N-1=2$ so that $N=3$.

Finally, we should add that some modern theories, like string theory, assume a higher number of spatial dimensions, where the extra dimensions are of a much smaller scale, such that, at common scales, the three dimensions of space are still a reasonable approximation.

File: content/2012-09-16-pyopencl-on-windows-without-a-gpu.md

Title: A PyOpenCL tutorial on Windows with or without a GPU
Tags: python, gpu

I've been using CUDA and
PyCUDA as
GPGPU platforms for a few years now.
They enable access to the incredible computational power of graphics cards
through a simple C-like language. A recent
Nvidia graphics card is nevertheless
required in order to execute CUDA code. Some computers may not include a
Nvidia GPU, but rather an
AMD/ATI card or even
an integrated graphics processor. Those computers thus cannot execute a CUDA
program.

Whereas changing the GPU on a desktop computer is straightforward (one still
needs to buy a new graphics card...), it is generally not possible to do so
on a laptop. It prevents a number of developers to run, compile or debug CUDA
code. It is also certainly a problem when distributing a software relying on
CUDA, since not having a CUDA-enabled Nvidia GPU prevents one to even launch
the program.

For these reasons, I've recently been interested in
OpenCL, a computing framework
equivalent to CUDA. The main difference is that OpenCL is an open standard
that has been adopted not only by Nvidia, but also by Intel and AMD. It means
that OpenCL programs may run on (recent) graphics card of either Nvidia or
AMD. In addition, a very interesting feature is that OpenCL programs may run
on a computer that does not even have a OpenCL-enabled GPU, but only a
recent-enough CPU. OpenCL code can be compiled for CPUs through the Intel
SDK for OpenCL
(Intel-only CPUs) or the AMD APP
SDK. OpenCL code is still
accelerated through vectorization instructions on CPUs (e.g.
SSE or
AVX) and multicore
CPUs.

This is really convenient since one can develop, debug and run an OpenCL
program on virtually any computer, even if it does not include an
OpenCL-enabled GPU. Moreover, distributing such a program is easier since the
end-user does not necessarily need to have a specific graphics card. Of
course, the performance of the program will be far less interesting than with
such a GPU, but the program will still run correctly.

Here I detail my first experience with OpenCL on Python (using
PyOpenCL).
The installation part is actually the less trivial part (while still being
relatively easy) and may be of interest to some people.

Installation of an OpenCL SDK

I'll explain how I could install OpenCL and PyOpenCL on an "old" laptop with
an ATI Radeon Mobility HD
3650
graphics card that does not support
OpenCL, an Intel Core 2 Duo CPU,
and a 32-bits Windows 8 operating
system.

First, one needs to install an OpenCL SDK. AMD and Intel have their own SDKs
and I installed both. Later, we'll see that PyOpenCL allows us to choose, at
compilation-time, which SDK to use. The installation of the SDKs is
straightforward on Windows.

Installation of PyOpenCL

PyOpenCL is to OpenCL what PyCUDA is to CUDA: a Python wrapper to those GPGPU
platforms. It is developed and maintained by Andreas
Klöckner.

Installing PyOpenCL on Windows is easy when using the binary package provided
by Christoph Gohlke. His
webpage contains Windows binary
installers for the most recent versions of hundreds of Python packages. It is
of invaluable help for those Python users that use Windows. It is also
possible yet much more complicated to build PyOpenCL from source on
Windows.

PyOpenCL has a few uncommon dependencies (beyond
Numpy), like
decorator and
PyTools (actually the latter includes
decorator, so I'm not even sure one needs to install decorator separately).
These libraries can be easily installed with the easy_install command-line
tool provided by the Distribute
library.

Some environment variables may need to be specified, like PYOPENCL_CTX or
PYOPENCL_COMPILER_OUTPUT (see below). In order to do this, one needs to go
to the advanced system parameters and add new variables in the User
variables section (see
here
for details).

In order to quickly test the installation of PyOpenCL, try the following
commands in a Python console:

text
import pyopencl
import pyopencl.array

We now turn to the execution of a very simple OpenCL program.

PyOpenCL Hello World script

Save the following code in a .py file and execute it. The OpenCL code should be
compiled and executed correctly, and the squares of integers should be
displayed. Some warnings may be displayed even if everything worked corectly.
In addition, PyOpenCL may require the user to decide which SDK to use if several
are installed. In order not to be asked everytime, one can set the
PYOPENCL_CTX environment variable to the corresponding index (see above).

I could use both Intel and AMD SDKs. Ensure that you run Python in a command
line as an administrator (otherwise it may cause issues with
compilation, at least that's what I could note on my Windows 8 setup).

The actual code is self-explanatory, especially for someone who already knows
CUDA. This program generates a Numpy array with all integers between 0 and 9,
passes it to an OpenCL kernel that computes the square of an array of integers,
dynamically compiles and executes this kernel, and copies back the output from
the context memory to Python.

text
# import PyOpenCL and Numpy. An OpenCL-enabled GPU is not required,
# OpenCL kernels can be compiled on most CPUs thanks to the Intel SDK for OpenCL
# or the AMD APP SDK.
import pyopencl as cl
import numpy as np

# create an OpenCL context
ctx = cl.create_some_context()
queue = cl.CommandQueue(ctx)

# create the kernel input
a = np.array(np.arange(10), dtype=np.int32)

# kernel output placeholder
b = np.empty(a.shape, dtype=np.int32)

# create context buffers for a and b arrays
# for a (input), we need to specify that this buffer should be populated from a
a_dev = cl.Buffer(ctx, cl.mem_flags.READ_ONLY | cl.mem_flags.COPY_HOST_PTR, 
                    hostbuf=a)
# for b (output), we just allocate an empty buffer
b_dev = cl.Buffer(ctx, cl.mem_flags.WRITE_ONLY, b.nbytes)

# OpenCL kernel code
code = """
__kernel void test1(__global int* a, __global int* b) {
    int i = get_global_id(0);
    b[i] = a[i]*a[i];
}
"""

# compile the kernel
prg = cl.Program(ctx, code).build()

# launch the kernel
event = prg.test1(queue, a.shape, a_dev, b_dev)
event.wait()

# copy the output from the context to the Python process
cl.enqueue_copy(queue, b, b_dev)

# if everything went fine, b should contain squares of integers
print(b)

Here is the output I get on my installation.

text
:::text
C:\Python27\lib\site-packages\pyopencl\__init__.py:32: CompilerWarning: Built kernel retrieved from cache. Original from-source build had warnings:
Build on  succeeded, but said:

Build started
Kernel <test1> was successfully vectorized
Done.
  warn(text, CompilerWarning)
C:\Python27\lib\site-packages\pyopencl\__init__.py:32: CompilerWarning: From-binary build succeeded, but resulted in non-empty logs:
Build on  succeeded, but said:

Build started
Kernel <test1> was successfully vectorized
Done.
  warn(text, CompilerWarning)
[ 0  1  4  9 16 25 36 49 64 81]

Final notes

There is of course far more to tell about OpenCL that what I've shown here.
The point was to show how one could use GPGPU on a standard computer that
does not have a GPGPU-enabled graphics card, by using OpenCL and Python.
Even if I don't have much experience in OpenCL yet, I think it has a promising
future as an open and portable concurrent to CUDA.

Here are some interesting related links:

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

#awesome Browse awesome catalog → #awesome-list Browse awesome-list catalog → #lecture-notes Browse lecture-notes catalog → #list Browse list catalog → #mathematics Browse mathematics catalog → ⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →