Apache SparkをJupiter Notebookからも使えるようにする

pip install nose "ipython[notebook]"

の後

PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook --no-browser --port=7777" pyspark

で起動

参照: http://blog.insightdatalabs.com/jupyter-on-apache-spark-step-by-step/

$ PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook --no-browser --port=7777" pyspark                                                     
[TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook` in the future
[I 12:19:16.840 NotebookApp] Loading IPython parallel extension
[I 12:19:16.847 NotebookApp] Serving notebooks from local directory: /Users/funami/src/kaizen-issue-finder
[I 12:19:16.847 NotebookApp] 0 active kernels 
[I 12:19:16.847 NotebookApp] The Jupyter Notebook is running at: http://localhost:7777/?token=92cacf69db34b4a1506a10f7a8ab855f05d61ce530b8786c
[I 12:19:16.848 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:19:16.848 NotebookApp] 
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:7777/?token=92cacf69db34b4a1506a10f7a8ab855f05d61ce530b8786c

PYSPARK_DRIVER_PYTHON=ipython はもう、古いらしい

PYSPARK_DRIVER_PYTHON=jupyter すると WARNINGでなくなる

 $ PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook --no-browser --port=7777" pyspark
[I 12:20:04.173 NotebookApp] Loading IPython parallel extension
[I 12:20:04.179 NotebookApp] Serving notebooks from local directory: /Users/funami/src/kaizen-issue-finder
[I 12:20:04.179 NotebookApp] 0 active kernels 
[I 12:20:04.179 NotebookApp] The Jupyter Notebook is running at: http://localhost:7777/?token=f39bd5b66eb8316b99edc92da2314e7ed075712b811631ee
[I 12:20:04.180 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:20:04.180 NotebookApp] 
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:7777/?token=f39bd5b66eb8316b99edc92da2314e7ed075712b811631ee