Pi _ Jupyter - dwisianto/dwisianto GitHub Wiki
- ipykernel
- ipynb
- nbmake
- nbsphinx
Jupyter in a Conda Environment
KernelSpec3
$ conda activate ml
(ml) $ conda install ipykernel
(ml) $ ipython kernel install --user --name=<any_name_for_kernel>
(ml) $ conda deactivate
- How To Install Jupyter Notebook And The Kernel In Your Conda Environmenthow-to-install-jupyter-notebook-and-the-kernel-in-your-conda-environment
- Assuming your conda-env is named ml, it is as simple as:
- conda install ipykernel installs all dependencies needed to use jupyter.
- ipython kernel install --user --name=<any_name_for_kernel> installs the kernel for this environment. I usually use the same kernel name as the environment name here (i.e. ml in this example).
- Run jupyter kernelspec list to get the paths of all your kernels.
- jupyter kernelspec list
- Then simply uninstall your unwanted-kernel:
- jupyter kernelspec uninstall unwanted-kernel
Kernelspec1
https://queirozf.com/entries/jupyter-kernels-how-to-add-change-remove
- jupyter kernelspec list
- jupyter kernelspec add
- jupyter kernelspec remove
- python -m ipykernel install --user --name mycustomkernel
Kernelspec2
How To Install Jupyter Notebook And The Kernel In Your Conda Environmentlink
Assuming your conda-env is named ml, it is as simple as:
$ conda activate ml
(ml) $ conda install ipykernel
(ml) $ ipython kernel install --user --name=<any_name_for_kernel>
(ml) $ conda deactivate
conda install ipykernel installs all dependencies needed to use jupyter.
ipython kernel install --user --name=<any_name_for_kernel> installs the kernel for this environment. I usually use the same kernel name as the environment name here (i.e. ml in this example).
Machine One
- Location
- cd d2/s2/c2/pj
- jupy.sh
- conda activate jupy39
- jupy.sh
#
# JUPY
#
export JUPY_HOME=/anaconda3/envs/jupy39/
export JUPY_WORK=~/d/s/m/wk/
alias jpl="cd $JUPY_WORK; $JUPY_HOME/bin/jupyter-lab"
echo " JUPY: JUPY_HOME $JUPY_HOME"
echo " JUPY_WORK $JUPY_WORK"
- Organization
- j22
- src
- jnb
- tst
- j22_
- dat
- j22-
- out
- tst
- j22
- Logging
- FAQ
- Skipping Certain Cells
- https://stackoverflow.com/questions/19309287/how-to-intermittently-skip-certain-cells-when-running-ipython-notebook
- Environment Variables
Convert Notebook to python script
one file
which jupyter-nbconvert
jupyter nbconvert --to python my_notebook.ipynb
all notebooks in the directory
pyt_exe="/mlstorage/mansjurd/d21/apps/ana/c202106a/envs/og21a/bin/pytest"
jupy_exe="/mlstorage/mansjurd/d21/apps/ana/c202106a/envs/og21a/bin/jupyter"
work_dir=""
echo "find . -name '*.ipynb' ! -name '*checkpoint.ipynb' -exec ${jupy_exe} nbconvert --to python {} \;"
find . -name '*.ipynb' ! -name '*checkpoint.ipynb' -exec ${jupy_exe} nbconvert --to python {} \;
pytest
Logging
One
"""
In jupyter notebook simple logging to console
"""
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
# Test
logger = logging.getLogger('LOGGER_NAME')
logger.debug('This is hidden')
logger.info('So this is shown on the console')
logger.warning('This too')
Two
"""
In jupyter notebook simple logging to console and file:
"""
import logging
import sys
logging.basicConfig(
level=logging.INFO,
format='[{%(filename)s:%(lineno)d} %(levelname)s - %(message)s',
handlers=[
logging.FileHandler(filename='tmp5a.log'),
logging.StreamHandler(sys.stdout)
]
)
# Test
logger = logging.getLogger('LOGGER_NAME')
logger.debug('This message should go to the log file and to the console')
logger.info('So should this')
logger.warning('And this, too')
Logging Three
"""
Setup simple logging in python. This logs info message to stdout and debug messages to file.
Sure it's long but this is as simple as I could make it for this outcome.
Note: We must set the root logger at DEBUG level, since it must be higher than it's children to pass them on.
Then set filehandler at debug and stream handler at info.
"""
import logging
import sys
import datetime
# To use differen't log level for file and console
timestamp = datetime.datetime.utcnow().strftime('%Y%m%d_%H-%M-%S')
filename=f'/tmp/tmp5a_{timestamp}.log'
formatter = logging.Formatter('[%(asctime)s] %(name)s {%(filename)s:%(lineno)d} %(levelname)s - %(message)s')
file_handler = logging.FileHandler(filename=filename)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)
stream_handler = logging.StreamHandler(sys.stdout)
stream_handler.setLevel(logging.INFO)
# The handlers have to be at a root level since they are the final output
logging.basicConfig(
level=logging.DEBUG,
format='[{%(filename)s:%(lineno)d} %(levelname)s - %(message)s',
handlers=[
file_handler,
stream_handler
]
)
# Test
logger = logging.getLogger('LOGGER_NAME')
logger.debug('This message should go to the log file')
logger.info('This should go to the stdout and file')
logger.warning('And this, too')
print('filelog', open(filename).read())