MacOS Catalina was released on October 7, 2019, and has been causing quite a stir for Anaconda users. Apple has decided that Anaconda’s default install location in the root folder is not allowed. It moves that folder into a folder on your desktop called “Relocated Items,” in the Security folder. If you’ve used the .pkg installer for Anaconda, this probably broke your Anaconda installation. Many users discuss the breakage at https://github.com/ContinuumIO/anaconda-issues/issues/10998.
There is unfortunately no simple fix for your current installation. Some people have recommended moving the Anaconda folder from the “Relocated Items” folder back to your home folder, followed by modifying PATH to point at the new location. This is a partial fix, and may restore some core functionality for you. Unfortunately, many files in a given conda environment contain hard-coded paths to their install path (AKA prefix). When you copy or move folders around, these hard-coded paths are not accurate, and programs in your environment might look for other things in the wrong place and break.
So, where to go from here? You have several options.
As Apple has announced, Mac OS Catalina now runs under its own read only file system: macOS Catalina runs on a dedicated, read-only system volume called Macintosh HD. This volume is completely separate from all other data to help prevent the accidental overwriting of critical operating system files. 1 Sounds great. I am experiencing a Python menu issue on both of my Mac's. One Mac has macOS Catalina and Python 3.7.4 via pyEnv with Tcl/Tk 8.6.9 and the other has macOS Catalina and Python 3.8.0. Via pyEnv and Tcl/Tk 8.6.9 and both area exhibiting the same behavior. This behavior did not exist before upgrading from macOS Mojave to macOS Catalina.
Reinstallation
Choose one:
- The older Python language, version 2.7, is being deprecated in macOS 10.15 Catalina and won't be included in macOS 10.16. The same goes for other UNIX scripting languages.
- Step 4: Install pyenv's recommended build dependencies. Pyenv installs Python by downloading and compiling it from source. The software we installed during Step 2 is critical to this process, but pyenv's documentation also recommends installing additional build dependencies when working on a Mac.
We have tested the currently available shell installers, and found them to work fine with Catalina. If you rely on the GUI installer instead, you’ll need to wait for the soon-to-be-released Anaconda 2019.10 installer, which will be signed and notarized in accordance with Apple’s more stringent policies.
Repair
This is experimental, but will allow you to fix your old installation and keep all of your old environments. To do this, you’ll use our self-contained prefix replacement tool at https://repo.anaconda.com/pkgs/misc/cpr-exec/cpr-0.1.1-osx-64.exe Note that although these files have a “.exe” file suffix, they are not Windows executables. We use the .exe suffix to indicate a standalone executable.
The experimental tool uses pyinstaller to make a standalone execution from code at https://github.com/conda/conda-prefix-replacement, which is a consolidated collection of the prefix detection and replacement code that is spread between conda and conda-build. We’ve added a novel feature to CPR for “rehoming” environments, which is the functionality needed to address the moved environment problem. Specifically, given the current (new) prefix, rehoming attempts to detect the old prefix from files within, and then adjust the detected files to have the new prefix value.
In steps:
1. At a terminal, download the file and make it executable:
2. Run the CPR tool to fix embedded paths:
Let’s say you started with an install to /Anaconda3, which Apple helpfully moved to “Relocated Items.” You have moved that folder to ~/anaconda3. Example commands for using CPR to fix this are:
Again, that finds your old prefix by looking in some files that are known to record the prefix in a readily parseable way. If that fails, you can still fix things as long as you know what the original path was:
The CPR command is recursive. Because environments default to the envs folder within your Anaconda[2|3] folder, CPR will fix all of your environments within your base prefix as well.
3. Re-run conda init to fix your conda shell command:
NOTE: if you use a shell other than bash, specify it in the conda init command:
4. Fix your ~/.conda/environments.txt file:
Open the ~/.conda/environments.txt file, and find/replace all instances of the old install location (e.g. /Anaconda3) with the new one (e.g. ~/anaconda3).
5. Find any remaining references to the old environment in other configuration files:
Many packages in Anaconda manage their own configuration files, typically in some subfolder of your home directory. You can use a tool like grep or ripgrep to find these files. You’ll then need to edit them and fix the paths as you did with the environments.txt file. An example grep command might be:
We hope this helps you get back on track.
Anaconda Perspectives
From Agriculture to Art, Four Unexpected Ways Data Science is Improving our WorldRead MoreAnaconda Perspectives
State of Data Science 2021: Becoming “Essential,” Though Untapped Potential RemainsRead MoreFor Practitioners
A Python Data Scientist’s Guide to the Apple Silicon TransitionCrack open the source code of any contemporary data journalism project, and you'll probably find some Python. The popular programming language is useful in a wide variety of journalism-related practices. From data cleaning and data analysis, to web scraping such as ProPublica's collection of Cook County Jail Data to web app development like the Baltimore Sun Public Salary Records project.
Python's simple, readable syntax and active, open-source community make it a great first language for anyone who wants to learn to code.
But how do you get Python? The process of setting everything up on your own desktop or laptop can get tricky, especially for newcomers to coding.
That's why I've put together this guide for anyone who wants to set up a Python development environment on their Mac.
Our aim is to strip the setup process down to the most essential steps and explain what's happening and why it's necessary. We'll also point out when you have the opportunity to further customize your development environment.
Without further delay, buckle up and let's get started!
A note to Windows users...
We're not ignoring you. We know you're out there, and we think you're awesome. You also deserve a slick Python setup, one that affords collaboration with your peers who've made different hardware choices. We'll be back soon with a similar how-to guide fit for you.
Step 0: Install any outstanding security/system updates
First, check your App Store icon for any red badges. If any of these nagging notifications are related to system or security updates, time to finally deal with them. It's OK. We all ignore these longer than we should.
Step 1: Open Terminal
All Macs ship with a built-in terminal emulator that's simply called 'Terminal'.
You'll find Terminal in your Applications folder under Utilities. You can also find it via the Spotlight search tool (type ⌘ + Spacebar, then start typing 'terminal').
This application is a throwback to a time before personal computers when people interacted with giant central mainframes via a simple hardware device, little more than a keyboard with a video display.
Like those early terminal devices, the Terminal application allows access to the command line interfaces (CLIs) of software installed on your computer. These are the advanced controls hidden behind the graphical user interfaces (GUIs) we normally use.
We can't point, click, swipe or otherwise work the command line in all the intuitive ways most of us prefer to use computers. Instead we type commands, then press RETURN, then the screen prints output (sometimes).
The methods for installing Python and setting up Python projects outlined below are available only via the command line. We'll provide the exact the commands you will need to execute, along with the proper arguments and options. You're encouraged to copy this input from the code snippet boxes, like this one:
Then paste them into your Terminal window. Also, don't forget to press RETURN in order to execute each command.
Go deeper...
Terminal will definitely get the job done. However, as you spend more time on the command line, you may want to liven things by changing your theme or otherwise customizing the experience.
If the range of customizations for Apple's Terminal (located under Preferences) feels too limited, you can switch to a third-party terminal emulator. Here are a couple of great, free options:
- iTerm2 has been the most popular choice for years, and it keeps getting better with each release.
- Hyper is the latest hotness. It's dazzling. It's extensible. And because it's built with Electron and other web technologies, it's cross-platform compatible.
Step 2: Install Xcode Command Line Tools
Xcode is Apple's software developer kit for creating apps on Apple devices (iPhones, iWatches, Macs, etc.) It contains a bunch of stuff you probably don't want, but a few things you need for your Python setup, including the GNU Compiler Collection (aka, GCC).
The subset of Xcode that's useful to us—the Xcode Command Line Tools—can be installed with this command:
You'll see a prompt like this:
Click 'Install' to continue. Note that this step will take several minutes, so please be patient.1
Step 3: Install Homebrew
Homebrew is the de facto (though, unofficial) package manager for Mac. It allows you to install software that isn't pre-installed on your Mac and isn't available in the App Store.
Here's the command to install homebrew:
The output should look like this:
As prompted, press RETURN. Once again, you'll be asked to type in your password.
PLOT TWIST: You had Python all along (but don't use it)
Python has come pre-installed on every Mac since 2012. You can check which version you have installed with the following command:
The output will look like Python 2.7.15. The numbers after the second dot will vary.
So why are we jumping through all these hoops?
Because this 'system' Python is tightly integrated with your Mac's operating system, and it's maintained by Apple. Any one of those system updates that occasionally pop up in your App Store could modify this installation of Python or other software that relies on it. You don't want to disrupt these automatic updates. You also don't want to write any code that relies on this outdated Python installation that's not under your control.
Instead, you need to install a newer version of Python in a different location that's isolated from your system Python but still discoverable by your command line.
We could use Homebrew to install another version of Python, and a lot of other guides will suggest this route. However, the best tool we've found for tackling this problem is pyenv.
With pyenv, you can:
- Install any exact version of Python
- Install multiple versions of Python, all in isolation from each other; and
- Switch between Python versions without any fuss.
Also, pyenv has a handy plugin to help you manage virtual environments for your Python projects, which we will discuss later in this guide.
Go deeper
You can find out more about pyenv, how it works and how to use it in Managing Multiple Python Versions With pyenv from RealPython.com.
Step 4: Install pyenv's recommended build dependencies
pyenv installs Python by downloading and compiling it from source. The software we installed during Step 2 is critical to this process, but pyenv's documentation also recommends installing additional build dependencies when working on a Mac.
To satisfy these recommendations, type in brew's install command followed by the list of recommended software packages:
Step 5: Install pyenv
We recommend installing pyenv via the automatic installer provided by the project's maintainers:
This will install pyenv and several of its most useful plugins.
Step 6: Initialize pyenv and pyenv-virtualenv in your shell
At the bottom of the pyenv-installer's output, you should see something like this:
This message is letting you know that, while pyenv finished installing, the commands are not yet available to use. That's because your shell—the program you interact with via Terminal that actually executes the commands you type in—still needs help finding it.
Your shell environment includes a variable called PATH that contains a list of all the directories where the command line can find and execute commands. Type in the following command:
And you'll see the full list of directories, delimited by a :.
When, for example, you type python on the command line and press RETURN, your shell will read this list of directories from left to right, looking in each one for a program named 'python'. If it finds that program, the shell will execute that program. If not, it will respond with command not found.
So now you need to add pyenv's location to the front of this list of directories. That's what this line does:
Then you need to initialize pyenv:
And pyenv-virtualenv:
You could invoke these commands one at a time from the command line, but the results would only apply in your current session. Once you close your terminal window, you'll lose these changes.
Instead, you need to add these three lines to a shell initialization file that runs each time you start a new session in your terminal. Look back at the output from pyenv-installer, and you'll see the name of the file you need to modify, which is determined by which shell your terminal is running.
Most likely, you'll need to modify .bashrc, which is one of the config files for Bourne-Again Shell (aka, bash). Bash has been the default shell on Macs since the early 2000s, but starting with Catalina, which Apple is due to publicly release this month, macOS is switching to Z Shell (aka, zsh). If you've already upgraded, then you'll need to modify .zshrc.
Either of these config files will be in your user account's home directory, which has the alias ~. Assuming you're on bash, you can open that file in a text editor:
You might see an error that looks like The file /Users/{your user name}/.bashrc does not exist. If this happens, double-check the name of the file mentioned in the warning printed by at the end of pyenv's installation script. It might recommend modifying ~/.bash_profile, ~/.profile, ~/.zshrc or the name of some other shell initialization file particular to your system's configuration.
If you're sure you have the right file name, but it doesn't exist, just create the file using the touch command:
If you still can't figure out which file to modify, check out the Unix shell initialization page in pyenv's wiki. It may also be time to reach out to your nearest sys admin.
Once you've opened the proper shell initialization file, copy these three lines:
Then paste them at bottom of the file, save your changes and close the text editor window.
Step 7: Restart your shell
The easiest way to do this is to close your current Terminal window, and open a new one.
You can then check if pyenv is working properly like so:
The output should look like this:
Step 8: Install the latest (stable) version of Python
Still with us? Now you can install whichever version of Python you want. Let's start by seeing what's available by passing the --list option to pyenv's install command:
As you'll see, there are many versions of Python and a variety of distributions of these versions. We recommend installing the official latest stable version. That would be 3.7.4 as of the original date this guide was published.
This step will take a while since pyenv builds Python from source with each installation.
You may notice that, as time passes, pyenv's list of Python versions available to install will become stale. You can update this list at any time using the update command, which is provided by another plugin in the pyenv-installer bundle:
Step 9: Switch your global Python
Your current 'global' Python installation is still system, as the pyenv global command will tell you:
Again, you don't want to muck around with the system's Python installation. So let's switch to the new version of Python we just installed:
You can also check which versions of Python you've previously installed by invoking pyenv's versions command:
Step 10: Set up a new Python project
By this point, you've fully switched over to the latest version of Python. We're now in the homestretch!
Just so you know: You likely won't ever need to repeat Steps 1 through 9, not until you buy a new computer at least.
In this final step, you'll set up a new Python project. This is a process you will repeat many times. After all, you're a prolific and unstoppable coder/journalist, and we expect you to keep on crushing it.
Each of your Python projects needs two things.
First, a Python project needs a directory (or folder) to hold all of your code and related assets. We suggest keeping each project directory inside another directory that's easy to find. For example, you might make a directory called 'Devel' (short for development) inside your home directory:
Now create a new project directory inside your directory of projects. Call it 'new-project':
Second, each Python project needs a virtual environment, which is just another directory that contains a Python installation, plus a bunch of other packages, all specific to that project. This allows you to keep the dependencies for each of your Python projects isolated from each other.
We can initialize a new virtual environment using the virtualenv-init command from the pyenv-virtualenv plugin:
Note that it's good practice to ensure your project's directory and virtual environment have the same name.
Then, we can connect our new Python project's directory to its virtual environment so that whenever we navigate into our project's directory, the virtual environment will be activated automatically.
First, invoke cd to change into your project's directory:
Then invoke the pyenv local command, passing in the virtual environment's name:
Notice how your command line's prompt changes to include the name of your project like (new-project). This is a visual indicator that the project's virtual environment is now active.
If you change back to your home directory:
The project's name will disappear from the prompt, indicating the virtual environment is now deactivated. Change back into the project's directory:
And the virtual environment is automatically reactivated. How convenient!
Now you can use pip, Python's standard package manager, to grab whatever third party packages your project needs.
For example, if this project were a web scraper, you would probably want:
- requests for handling HTTP requests; and
- bs4 (aka, Beautiful Soup) for parsing HTML content.
Additionally, we encourage new data journalists to work in Jupyter Notebooks because it allows you to mix code and prose together in the same document. You'll also appreciate the instant feedback that Jupyter provides for each code snippet you write, especially while you are still learning Python's syntax and the interfaces of the packages you import.
Go deeper...
Read Python Virtual Environments: A Primer from RealPython.org to learn more about the history behind virtual environments and the many tools available for managing them.
You did it!
Thanks for sticking with us through this journey. We hope you've found this guide helpful and informative. Now go forth and do great work!
An early version of this article directed users on macOS Mojave (10.14) to install additional Apple SDK headers. However, with changes introduced in more recent versions of the Xcode command line tools, this step is no longer necessary.
Senior EditorJames Gordon researched and developed new platforms for journalism, drawing on his background in coding, data analysis and product management.READ NEXT
Gimbals for video: B...
Tags
codingdatahow-toinnovationinnovation labjames gordonkat duncanpythonscrapingterminal
Share
RelatedStories
- News
- Series
- Events
- Newsletters
Python Idle Mac Catalina
Accountable JournalismApplication Development LabInnovation TeamPictures of the YearRJI FellowshipsRJI Student Innovation CompetitionProjectsImpact with Context: Tool to Measure News ImpactJournalism Digital News Archive (JDNA)Potter Digital AmbassadorsIs Python Available For Mac
Trusting NewsJournalism Institute
Python Download For Mac Catalina
Administrative Offices, Suite 300
Columbia, MO 65211
573-884-9121 | Fax: 573-884-3824
rji@rjionline.org
Python Download For Mac Catalina
Copyright © 2021 - Curators of the University of Missouri. All rights reserved.
Python Os X Catalina
DMCA and other copyright information. An equal opportunity/affirmative action institution. Privacy Policy. Terms of Service.