These are notes for getting the Charm library working on an Android device.
NB: Move ahead to this section if you just want to install the final APKs on a device.
Preliminaries
Start by putting everything in a new, fresh directory, named "fresh". All the instructions below will refer to that.
Working installations of git, Mercurial, Eclipse, the Android SDK, and the Android NDK are required. Setup an Android cross compilation toolchain, as below, and make sure all the required executables are inside the PATH.
Compiling Python for Android (Py4A)
The first step is to get Python working on Android. The Python for Android project already has most of the work done. Start off by getting the code, as below:
It may take a while, as the repository is large. Once that's done, build it to make sure it's working. First, however, there are some modifications that need to be done to the configuration.
In the file ~/fresh/python-for-android/python3-alpha/python3-src/xbuild.sh
(right below the popd line at the top), add the below lines:
Do a clean build:
The build should complete successfully, and the following .zip files should be in the output folder, ~/fresh/python-for-android/python3-alpha/python3-src/
:
- python3_extras_r11.zip
- python3_r10.zip
- python3_scripts_r11.zip
Preparing Charm's Dependencies
Charm has a few dependencies, which must be built and prepared before Charm itself can be built.
PyParsing
First, start off with PyParsing. Download the version required by Charm, which is 1.5.5 at the time of this writing, from this link. Open the tarball, and from inside the folder, extract the pyparsing_py3.py
file and save it as ~/fresh/python-for-android/python3-alpha/extra_modules/pyparsing.py
. (NB: Note the change in filename.)
pkg_resources
As part of the egg importing process, we need to use the pkg_resources module from distribute. Download the tarball from this link. Open the tarball, and from inside the folder, extract the pkg_resources.py
file and save it into ~/fresh/python-for-android/python3-alpha/extra_modules/
. This was written for python2, so we need to convert it to python3:
OpenSSL
Charm needs to link against the Android version of openssl that's deployed on the device/emulator that's being run. The easiest way to do this for now is to just fetch the binaries and use those (assuming a connection to the device exists, using adb), as below:
GMP
- Start off by getting GMP here.
- Extract the folder inside of
~/fresh/python-for-android/python3-alpha/
. - Copy the
xbuild.sh
andxconf.sh
files from~/fresh/python-for-android/python3-alpha/sqlite3/
to~/fresh/python-for-android/python3-alpha/gmp-5.0.2/
. - Replace the
config.sub
andconfig.guess
files inside~/fresh/python-for-android/python3-alpha/gmp-5.0.2/
with updated versions from here.
Now, compile it:
Upon a successful build, libgmp.so
should be present inside ~/fresh/python-for-android/python3-alpha/thirdparty/lib/
PBC
Start off by getting PBC here. Extract the folder inside of ~/fresh/python-for-android/python3-alpha/
. Replace the config.sub
and config.guess
files inside ~/fresh/python-for-android/python3-alpha/pbc-0.5.12/
with updated versions from here.
PBC has one slight incompatibility that needs to be fixed. Replace the ~/fresh/python-for-android/python3-alpha/pbc-0.5.12/misc/extend_printf.c
file with this one, as per instructions in this thread.
The next step is to tell PBC where to find GMP. Add two files as below. Note that the paths used are absolute paths, the ~ does not get expanded, so the full home directory must be specified.
In ~/fresh/python-for-android/python3-alpha/pbc-0.5.12/xbuild.sh
:
In ~/fresh/python-for-android/python3-alpha/pbc-0.5.12/xconf.sh
:
Now, compile it:
Upon a successful build, libpbc.so
should be present inside ~/fresh/python-for-android/python3-alpha/thirdparty/lib/
Integration with Py4A
The next step is to integrate the building of these modules with the python-for-android build. PyParsing is already handled by just placing the file inside the required directory. OpenSSL needs no work. For the others, some work is required.
Start off by making the following additions.
In ~/fresh/python-for-android/python3-alpha/buildall.sh
: (above "Python 3")
In ~/fresh/python-for-android/python3-alpha/cleanall.sh
: (above "Python 3")
In ~/fresh/python-for-android/python3-alpha/python3-src/xpack.sh
: (above "#Symbolic linked libs just take up space")
Do a clean build:
Verify the build by checking for the existence of pyparsing.py
and pkg_resources.py
inside ~/fresh/python-for-android/python3-alpha/python3-src/python3_extras_r11.zip
; and for the existence of libpbc.so.1
and libgmp.so.10
inside ~/fresh/python-for-android/python3-alpha/python3-src/python3_r10.zip
.
Building Charm
Prerequisites
Make sure a valid installation of Python3 and setuptools is available. On Ubuntu, the following command should do it:
Fetch the Charm source:
Modifications to the Charm Source
NOTE: The following changes have already been done in the repository mentioned above. They are just listed here for reference.
The integermath extension in Charm uses a function which is deprecated in later versions of OpenSSL. The Ubuntu versions have that left in for backward compatibility, however it's not available on the Android version. So, the following modification is needed:
Similarly, it has a workaround for the paths used in importing, which isn't really needed on Android. Modify that as well:
Lastly, we need to modify setup.py to create an egg, as that's what Py4A requires. Replace the whole of that file with the below:
Creating the egg
Now, the egg can finally be created (it should appear in the dist folder if it succeeds):
Integration with Py4A
This is fairly simple. First, copy the egg over, as below:
Now, add another entry to the ~/fresh/python-for-android/python3-alpha/python3-src/xpack.sh
script so that it copies over the benchmark.so
file into the appropriate folder. Add this line right above the "#Symbolic linked libs just take up space" line:
The benchmark.so file needs to be copied over because of peculiarities with the way Charm loads that module. NB: This should probably be fixed upstream.
Now we just need to add an entry so that the egg can be properly loaded. Create the file ~/fresh/python-for-android/python3-alpha/python3-src/Lib/site-packages/easy-install.pth
with the following contents:
Lastly, for testing purposes, add in a script from the Charm library so that it's possible to run it on the target environment.
Now, run a clean build of Py4A:
To confirm that the build worked, check for the following three things:
- The Charm egg should appear inside the site-packages folder in python3_extras_r11.zip
- The dabe_aw11.py script should appear inside python3_scripts_r11.zip
- The benchmark.so file should appear inside /python3/lib/python3.2/lib-dynload/ in python3_r10.zip
Deploying to the device
Preliminaries
Py4A requires the files we have created previously to be available on some web server, as part of the installation process. Upload the following files to some server that the target device can access:
~/fresh/python-for-android/python3-alpha/VERSIONS
~/fresh/python-for-android/python3-alpha/python3-src/python3_extras_r11.zip
~/fresh/python-for-android/python3-alpha/python3-src/python3_scripts_r11.zip
~/fresh/python-for-android/python3-alpha/python3-src/python3_r10.zip
The following descriptions will use the http://drop.mhlakhani.com URL, replace that with your own as appropriate.
Creating the APK
Start with a fresh workspace, and import the following projects from the ~/fresh/python-for-android/android/
directory:
- Common
- InterpreterForAndroid
- Utils
- Python3ForAndroid
For each project, the ANDROID_SDK path variable (inside Properties->Java Build Path->Libraries) needs to be set so that it builds correctly. Also change the target to API 10 (Android 2.3.3).
For the Python3ForAndroid project, first set the build target to API 10 (Properties->Android), and add the other projects, as well as the code for this project, into the exported entries (Properties->Java Build Path->Order and Export). Do a clean build, and make sure it builds fine.
Now there are a few modifications that need to be made. First, we need to update the URL for the files; and we need to make sure the egg cache is writable so that the Charm egg works properly. The following diff shows the changes that need to be done:
After this modification is made, do a clean build, and then export a signed APK using Android Tools->Export Signed Application Package. Place the .apk in ~/fresh
Final Deployment
Py4A runs under SL4A. Download that APK from here and place it in ~/fresh
If you just want to run Charm without having to go through the whole build process, you can also grab a pre-prepared version of the APK from this link.
For testing purposes, use an emulator image for Android 10 (these steps could potentially be run on a hardware device too). Create an appropriate image, and launch the emulator. Using adb, install the applications, as below:
Now, in the emulator, open the Python3ForAndroid application, and click the "Install" button at the top. It will download the previously created zip files and install them; the process may take a while.
Once that is done, open the SL4A application, and run the dabe_aw11.py
script. It should run correctly, and show a "Successful Decryption!" message.