This video tutorial was created as a guide to setup Jupyter notebook to run R and Python with Apache Spark on Windows OS, there are more steps than create similar hands on environment on MacOS and Linux. If this tutorial is helpful, please like it and subscribe. Thanks. Commands used in the video are following (please expand):
If you do not already have Python 3.6 or better, you can download and install form:
www.python.org...
Check if pip is already installed, if not, install pip from:
pip.pypa.io/en...
Install virtualenv
pip install virtualenv
Create virtual environment
python -m virtualenv myspark
Activate virtual environment (Mac or Linux):
./myspark/bin/activate
Activate virtual environment (Windows):
.\myspark\scripts\activate
Install notebook within virtual environment:
pip install notebook
Make sure JDK 17 has been setup on your computer, if not you need to download and setup JDK 17
To download JDK 17:
www.oracle.com...
Setup environment variable:
JAVA_HOME and add JAVE_HOME/bin to the path
Download and Setup Spark from:
spark.apache.o...
Download Spark compress tar file (tgz), expand and rename the expanded folder to spark
Setup environment variable SPARK_HOME to the path of spark folder
Add SPARK_HOME/bin to the path
Install R and Rstudio:
posit.co/downl...
Then Launch R in command line window, run following commands under R command prompt:
install.packages("usethis",repos = "cran.us.r-proje...")
install.packages("pkgdown",repos = "cran.us.r-proje...")
print("Install devtools begin")
install.packages("devtools",repos = "cran.us.r-proje...")
print("Install IRkernel begin")
devtools::install_github("IRkernel/IRkernel", force = TRUE)
system.file('kernelspec', package = 'IRkernel')
print("kernalspec begin")
IRkernel::installspec()
9 окт 2024
