Python Libraries for Health Data Analysis: A Review for Apple Silicon

Utharn Buranasaksee, Nadaphast Koomklang, Namooy Panya, Montri Sangthong, Ekachai Naowanich

Abstract


The paper presents an overview of health data analysis and the challenges associated with handling large and complex datasets. Then, the authors highlight the significance of Python as a popular language for health data analysis, emphasizing its flexible syntax and efficient libraries written in C/C++. As the data keep larger, this raises memory-related issues in current hardware platforms. While the emergence of the Arm platform, exemplified by Apple's M1 and its predecessors, the Arm platform is presented as a potential solution providing that shared memory and high-speed memory access are built into the platform. Therefore, many Python libraries for health data analysis are surveyed and analyzed for compatibility issues with Apple silicon. Finally, we conclude the migration checklist for developers transitioning to the Arm platform.

Full Text:

PDF

References


K. J. Millman and M. Aivazis, “Python for scientists and engineers,” Comput Sci Eng, vol. 13, no. 2, pp. 9–12, Mar. 2011, doi: 10.1109/MCSE.2011.36.

A. Ilievski, V. Zdraveski, and M. Gusev, “How CUDA Powers the Machine Learning Revolution,” 2018 26th Telecommunications Forum, TELFOR 2018 - Proceedings, 2018, doi: 10.1109/TELFOR.2018.8611982.

Apple Inc., “Apple unleashes M1 - Apple,” Nov. 10, 2020. https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/ (accessed Aug. 08, 2023).

D. L. F. Lam, “Study of iPhone’s Big Data, Market Share, Usage and Their Relationships,” ACM International Conference Proceeding Series, pp. 7–10, Aug. 2020, doi: 10.1145/3421537.3421542.

X. Wu, P. Brazzle, and S. Cahoon, “Performance and Energy Consumption of Parallel Machine Learning Algorithms,” 2023.

Apple Inc., “Apple unveils M2 Pro and M2 Max: next-generation chips for next-level workflows - Apple,” Jan. 17, 2023. https://www.apple.com/newsroom/2023/01/apple-unveils-m2-pro-and-m2-max-next-generation-chips-for-next-level-workflows/ (accessed Aug. 08, 2023).

I. Stancin and A. Jovic, “An overview and comparison of free Python libraries for data mining and big data analysis,” 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics, MIPRO 2019 - Proceedings, pp. 977–982, May 2019, doi: 10.23919/MIPRO.2019.8757088.

S. Raschka, J. Patterson, and C. Nolet, “Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence,” Information 2020, Vol. 11, Page 193, vol. 11, no. 4, p. 193, Apr. 2020, doi: 10.3390/INFO11040193.

J. Siebert, J. Groß, and C. Schroth, “A systematic review of Python packages for time series analysis,” Apr. 2021, Accessed: Aug. 08, 2023. [Online]. Available: https://arxiv.org/abs/2104.07406v2

NumPy Developers, “NumPy 1.21.1 Release Notes — NumPy v2.0.dev0 Manual.” https://numpy.org/devdocs/release/1.21.1-notes.html (accessed Aug. 09, 2023).

Github Inc., “Can’t install Pandas on Mac M1 · Issue #40611 · pandas-dev/pandas,” Mar. 24, 2021. https://github.com/pandas-dev/pandas/issues/40611 (accessed Aug. 09, 2023).

Github Inc., “ENH: Please provide ‘universal2’ wheels for macOS · Issue #39053 · pandas-dev/pandas,” Jan. 21, 2022. https://github.com/pandas-dev/pandas/issues/39053 (accessed Aug. 09, 2023).

Github Inc., “matplotlib 3.4.2 using on M1 Mac, with python 3.9.4 · Issue #20261 · matplotlib/matplotlib,” May 20, 2021. https://github.com/matplotlib/matplotlib/issues/20261 (accessed Aug. 09, 2023).

Github Inc., “Build wheels for Apple Silicon. by QuLogic · Pull Request #20970 · matplotlib/matplotlib,” Sep. 02, 2021. https://github.com/matplotlib/matplotlib/pull/20970 (accessed Aug. 09, 2023).

“scipy · PyPI.” https://pypi.org/project/scipy/1.7.3/ (accessed Aug. 09, 2023).

Github Inc., “Unable to install dependencies on Mac M1 · Issue #22 · GokuMohandas/mlops-course,” Feb. 13, 2023. https://github.com/GokuMohandas/mlops-course/issues/22 (accessed Aug. 09, 2023).

Github Inc., “BUG: optimize.brentq triggers an overflow error · Issue #14851 · scipy/scipy,” Oct. 14, 2021. https://github.com/scipy/scipy/issues/14851 (accessed Aug. 09, 2023).

P. J. A. Cock et al., “Biopython: Freely available Python tools for computational molecular biology and bioinformatics,” Bioinformatics, vol. 25, no. 11, pp. 1422–1423, Jun. 2009, doi: 10.1093/bioinformatics/btp163.

F. Di Gregorio and D. Varrazzo, “Psycopg – PostgreSQL database adapter for Python — Psycopg 2.9.7 documentation.” https://www.psycopg.org/docs/ (accessed Aug. 09, 2023).

Github Inc., “PyMySQL/mysqlclient: MySQL database connector for Python (with Python 3 support).” https://github.com/PyMySQL/mysqlclient (accessed Aug. 09, 2023).

Github Inc., “pydicom/pydicom: Read, modify and write DICOM files with python code.” https://github.com/pydicom/pydicom (accessed Aug. 09, 2023).

Mislav. Grgić, Sonja. Grgić, Institute of Electrical and Electronics Engineers. Region 8., Institute of Electrical and Electronics Engineers. Croatia Section., and S. and I. Processing. European Association for Speech, "Proceedings ELMAR-2008 : 50th International Symposium ELMAR-2008, 10-12 September 2008, Zadar, Croatia," Croatian Society Electronics in Marine, 2008, p. 654.

R. A. Poldrack et al., “Towards open sharing of task-based fMRI data: The OpenfMRI project,” Front Neuroinform, vol. 7, no. JUNE, Jun. 2013, doi: 10.3389/FNINF.2013.00012.

Github Inc., “[BUG] cannot install mkl (doc dependency) on Mac M1 Sillicon chip · Issue #3849 · nilearn/nilearn,” Jul. 20, 2023. https://github.com/nilearn/nilearn/issues/3849 (accessed Aug. 09, 2023).

Apple Inc., “Tensorflow Plugin - Metal - Apple Developer.” https://developer.apple.com/metal/tensorflow-plugin/ (accessed Aug. 09, 2023).

Github Inc., “Lack of Documentation for GPU Use, Especially in Metal GPUs in MacBook M1, M1 Max and M2 Chips · Issue #61092 · tensorflow/tensorflow,” Jun. 27, 2023. https://github.com/tensorflow/tensorflow/issues/61092 (accessed Aug. 09, 2023).

Apple Inc., “Accelerated PyTorch training on Mac - Metal - Apple Developer.” https://developer.apple.com/metal/pytorch/ (accessed Aug. 09, 2023).

The Linux Foundation, “MPS backend — PyTorch 2.0 documentation.” https://pytorch.org/docs/stable/notes/mps.html (accessed Aug. 09, 2023).

Lambda Inc., “GPU Benchmarks for Deep Learning | Lambda.” https://lambdalabs.com/gpu-benchmarks (accessed Aug. 09, 2023).

The Linux Foundation, “Introducing Accelerated PyTorch Training on Mac | PyTorch.” https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/ (accessed Aug. 09, 2023).


Refbacks

  • There are currently no refbacks.