Skip to content

wattsworth / puppet

Go to a project
Commit 9a73c107 by source_reader
Options

added support for jupyter and subdomain apps in nginx

parent a1c1c7c0
Inline Side-by-side
Showing with 634 additions and 5 deletions
README
...@@ -12,7 +12,7 @@ data=journal ...@@ -12,7 +12,7 @@ data=journal
3.) Run as a container 3.) Run as a container
sudo systemd-nspawn -D wattsworth_cubic/squashfs-root/ --bind /home/jdonnal/puppet:/home/ubuntu/puppet -b --network-interface=enxd8eb97bdd8ed --bind /media/jdonnal/wattsworth_tests:/home/ubuntu/tests sudo systemd-nspawn -D ~/ww_image_builder/cubic/squashfs-root/ --bind /home/jdonnal/puppet:/home/ubuntu/puppet -b --network-interface=enxd8eb97bdd8ed
4.) From inside the container run boostrap and apply scripts 4.) From inside the container run boostrap and apply scripts
......
index.html 0 → 100644
<!DOCTYPE HTML>
<html>
<head>
<meta charset="utf-8">
<title>Jupyter Notebook</title>
<link id="favicon" rel="shortcut icon" type="image/x-icon" href="/static/base/images/favicon.ico?v=97c6417ed01bdc0ae3ef32ae4894fd03">
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<link rel="stylesheet" href="/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=3c2a865c832a1322285c55c6ed99abb2" type="text/css" />
<link rel="stylesheet" href="/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=9df10041c3e07da766e7c48dd4c35e4a" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="/static/style/style.min.css?v=2165fc0d023f0baf5cce3b2a6db40e22" type="text/css"/>
<link rel="stylesheet" href="/static/auth/css/override.css?v=ae668279c8a80c1d2e81fc28345169ee" type="text/css" />
<link rel="stylesheet" href="/custom/custom.css" type="text/css" />
<script src="/static/components/es6-promise/promise.min.js?v=f004a16cb856e0ff11781d01ec5ca8fe" type="text/javascript" charset="utf-8"></script>
<script src="/static/components/react/react.production.min.js?v=34f96ffc962a7deecc83037ccb582b58" type="text/javascript"></script>
<script src="/static/components/react/react-dom.production.min.js?v=b14d91fb641317cda38dbc9dbf985ab4" type="text/javascript"></script>
<script src="/static/components/create-react-class/index.js?v=94feb9971ce6d26211729abc43f96cd2" type="text/javascript"></script>
<script src="/static/components/requirejs/require.js?v=951f856e81496aaeec2e71a1c2c0d51f" type="text/javascript" charset="utf-8"></script>
<script>
require.config({
urlArgs: "v=20200124150702",
baseUrl: '/static/',
paths: {
'auth/js/main': 'auth/js/main.min',
custom : '/custom',
nbextensions : '/nbextensions',
kernelspecs : '/kernelspecs',
underscore : 'components/underscore/underscore-min',
backbone : 'components/backbone/backbone-min',
jed: 'components/jed/jed',
jquery: 'components/jquery/jquery.min',
json: 'components/requirejs-plugins/src/json',
text: 'components/requirejs-text/text',
bootstrap: 'components/bootstrap/dist/js/bootstrap.min',
bootstraptour: 'components/bootstrap-tour/build/js/bootstrap-tour.min',
'jquery-ui': 'components/jquery-ui/jquery-ui.min',
moment: 'components/moment/min/moment-with-locales',
codemirror: 'components/codemirror',
termjs: 'components/xterm.js/xterm',
typeahead: 'components/jquery-typeahead/dist/jquery.typeahead.min',
},
map: { // for backward compatibility
"*": {
"jqueryui": "jquery-ui",
}
},
shim: {
typeahead: {
deps: ["jquery"],
exports: "typeahead"
},
underscore: {
exports: '_'
},
backbone: {
deps: ["underscore", "jquery"],
exports: "Backbone"
},
bootstrap: {
deps: ["jquery"],
exports: "bootstrap"
},
bootstraptour: {
deps: ["bootstrap"],
exports: "Tour"
},
"jquery-ui": {
deps: ["jquery"],
exports: "$"
}
},
waitSeconds: 30,
});
require.config({
map: {
'*':{
'contents': 'services/contents',
}
}
});
// error-catching custom.js shim.
define("custom", function (require, exports, module) {
try {
var custom = require('custom/custom');
console.debug('loaded custom.js');
return custom;
} catch (e) {
console.error("error loading custom.js", e);
return {};
}
})
document.nbjs_translations = {"domain": "nbjs", "locale_data": {"nbjs": {"": {"domain": "nbjs"}}}};
document.documentElement.lang = navigator.language.toLowerCase();
</script>
</head>
<body class=""
dir="ltr">
<noscript>
<div id='noscript'>
Jupyter Notebook requires JavaScript.<br>
Please enable it to proceed.
</div>
</noscript>
<div id="header" role="navigation" aria-label="Top Menu">
<div id="header-container" class="container">
<div id="ipython_notebook" class="nav navbar-brand"><a href="/tree" title='dashboard'>
<img src='/static/base/images/logo.png?v=641991992878ee24c6f3826e81054a0f' alt='Jupyter Notebook'/>
</a></div>
</div>
<div class="header-bar"></div>
</div>
<div id="site">
<div id="ipython-main-app" class="container">
<div class="row">
<div class="navbar col-sm-8">
<div class="navbar-inner">
<div class="container">
<div class="center-nav">
<form action="/login?next=%2Ftree%3F" method="post" class="navbar-form pull-left">
<input type="hidden" name="_xsrf" value="2|05940a82|5d1a0bb5443108ed90676fa5a48e4d4d|1579878511"/>
<label for="password_input"><strong>Password or token:</strong></label>
<input type="password" name="password" id="password_input" class="form-control">
<button type="submit" class="btn btn-default" id="login_submit">Log in</button>
</form>
</div>
</div>
</div>
</div>
</div>
<div class="col-sm-6 col-sm-offset-3 text-left rendered_html">
<h3>
Token authentication is enabled
</h3>
<p>
If no password has been configured, you need to open the notebook
server with its login token in the URL, or paste it above.
This requirement will be lifted if you
<b><a href='https://jupyter-notebook.readthedocs.io/en/stable/public_server.html'>
enable a password</a></b>.
</p>
<p>
The command:
<pre>jupyter notebook list</pre>
will show you the URLs of running servers with their tokens,
which you can copy and paste into your browser. For example:
</p>
<pre>Currently running servers:
http://localhost:8888/?token=c8de56fa... :: /Users/you/notebooks
</pre>
<p>
or you can paste just the token value into the password field on this
page.
</p>
<p>
See
<b><a
href='https://jupyter-notebook.readthedocs.io/en/stable/public_server.html'>
the documentation on how to enable a password</a>
</b>
in place of token authentication,
if you would like to avoid dealing with random tokens.
</p>
<p>
Cookies are required for authenticated access to notebooks.
</p>
<h3>Setup a Password</h3>
<p> You can also setup a password by entering your token and a new password
on the fields below:</p>
<form action="/login?next=%2Ftree%3F" method="post" class="">
<input type="hidden" name="_xsrf" value="2|05940a82|5d1a0bb5443108ed90676fa5a48e4d4d|1579878511"/>
<div class="form-group">
<label for="token_input"><h4>Token</h4></label>
<input type="password" name="password" id="token_input" class="form-control">
</div>
<div class="form-group">
<label for="new_password_input"><h4>New Password</h4></label>
<input type="password" name="new_password" id="new_password_input" class="form-control" required>
</div>
<div class="form-group">
<button type="submit" class="btn btn-default" id="login_new_pass_submit">Log in and set new password</button>
</div>
</form>
</div>
</div>
</div>
<script type="text/javascript">
require(["auth/js/main"], function (auth) {
auth.login_main();
});
</script>
<script type='text/javascript'>
function _remove_token_from_url() {
if (window.location.search.length <= 1) {
return;
}
var search_parameters = window.location.search.slice(1).split('&');
for (var i = 0; i < search_parameters.length; i++) {
if (search_parameters[i].split('=')[0] === 'token') {
// remote token from search parameters
search_parameters.splice(i, 1);
var new_search = '';
if (search_parameters.length) {
new_search = '?' + search_parameters.join('&');
}
var new_url = window.location.origin +
window.location.pathname +
new_search +
window.location.hash;
window.history.replaceState({}, "", new_url);
return;
}
}
}
_remove_token_from_url();
</script>
</body>
</html>
\ No newline at end of file
modules/docs/files/JUPYTER_README 0 → 100644
Jupyter is a web based Python development environment. For full
documentation see https://jupyter.org/. By default Jupyter is
configured to listen for connections from any interface on port 8888 (access at
http://localhost:8888).
**NOTE: YOU MUST CONFIGURE A PASSWORD TO USE JUPYTER**
Create or update the Jupyter password with the following command:
$> sudo jupyter-set-password
--- (Optional) Access through the Wattsworth Interface ---
To access Jupyter as a Data App add it to the proxy
configuration in [/etc/joule/main.conf]:
For example:
jupyter=http://127.0.0.1:8888
**NOTE: TO USE JUPYTER THROUGH WATTSWORTH (LUMEN)
YOU MUST CHANGE THE NGINX CONFIGURATION
TO SUPPORT SUBDOMAIN APPS**
See comments in [/etc/nginx/nginx.conf] for details
modules/docs/manifests/init.pp
...@@ -43,5 +43,11 @@ class docs { ...@@ -43,5 +43,11 @@ class docs {
group => $owner, group => $owner,
source => 'puppet:///modules/docs/STORAGE_README' source => 'puppet:///modules/docs/STORAGE_README'
} }
file{"${docs_path}/JUPYTER_README":
ensure => present,
owner => $owner,
group => $owner,
source => 'puppet:///modules/docs/JUPYTER_README'
}
} }
modules/jupyter/Gemfile 0 → 100644
source ENV['GEM_SOURCE'] || 'https://rubygems.org'
puppetversion = ENV.key?('PUPPET_VERSION') ? ENV['PUPPET_VERSION'] : ['>= 3.3']
gem 'metadata-json-lint'
gem 'puppet', puppetversion
gem 'puppetlabs_spec_helper', '>= 1.2.0'
gem 'puppet-lint', '>= 1.0.0'
gem 'facter', '>= 1.7.0'
gem 'rspec-puppet'
# rspec must be v2 for ruby 1.8.7
if RUBY_VERSION >= '1.8.7' && RUBY_VERSION < '1.9'
gem 'rspec', '~> 2.0'
gem 'rake', '~> 10.0'
else
# rubocop requires ruby >= 1.9
gem 'rubocop'
end
modules/jupyter/README.md 0 → 100644
# jupyter
#### Table of Contents
1. [Description](#description)
1. [Setup - The basics of getting started with jupyter](#setup)
* [What jupyter affects](#what-jupyter-affects)
* [Setup requirements](#setup-requirements)
* [Beginning with jupyter](#beginning-with-jupyter)
1. [Usage - Configuration options and additional functionality](#usage)
1. [Reference - An under-the-hood peek at what the module is doing and how](#reference)
1. [Limitations - OS compatibility, etc.](#limitations)
1. [Development - Guide for contributing to the module](#development)
## Description
Start with a one- or two-sentence summary of what the module does and/or what
problem it solves. This is your 30-second elevator pitch for your module.
Consider including OS/Puppet version it works with.
You can give more descriptive information in a second paragraph. This paragraph
should answer the questions: "What does this module *do*?" and "Why would I use
it?" If your module has a range of functionality (installation, configuration,
management, etc.), this is the time to mention it.
## Setup
### What jupyter affects **OPTIONAL**
If it's obvious what your module touches, you can skip this section. For
example, folks can probably figure out that your mysql_instance module affects
their MySQL instances.
If there's more that they should know about, though, this is the place to mention:
* A list of files, packages, services, or operations that the module will alter,
impact, or execute.
* Dependencies that your module automatically installs.
* Warnings or other important notices.
### Setup Requirements **OPTIONAL**
If your module requires anything extra before setting up (pluginsync enabled,
etc.), mention it here.
If your most recent release breaks compatibility or requires particular steps
for upgrading, you might want to include an additional "Upgrading" section
here.
### Beginning with jupyter
The very basic steps needed for a user to get the module up and running. This
can include setup steps, if necessary, or it can be an example of the most
basic use of the module.
## Usage
This section is where you describe how to customize, configure, and do the
fancy stuff with your module here. It's especially helpful if you include usage
examples and code samples for doing things with your module.
## Reference
Users need a complete list of your module's classes, types, defined types providers, facts, and functions, along with the parameters for each. You can provide this list either via Puppet Strings code comments or as a complete list in this Reference section.
* If you are using Puppet Strings code comments, this Reference section should include Strings information so that your users know how to access your documentation.
* If you are not using Puppet Strings, include a list of all of your classes, defined types, and so on, along with their parameters. Each element in this listing should include:
* The data type, if applicable.
* A description of what the element does.
* Valid values, if the data type doesn't make it obvious.
* Default value, if any.
## Limitations
This is where you list OS compatibility, version compatibility, etc. If there
are Known Issues, you might want to include them under their own heading here.
## Development
Since your module is awesome, other users will want to play with it. Let them
know what the ground rules for contributing are.
## Release Notes/Contributors/Etc. **Optional**
If you aren't using changelog, put your release notes here (though you should
consider using changelog). You can also add any additional sections you feel
are necessary or important to include here. Please use the `## ` header.
modules/jupyter/Rakefile 0 → 100644
require 'puppetlabs_spec_helper/rake_tasks'
require 'puppet-lint/tasks/puppet-lint'
require 'metadata-json-lint/rake_task'
if RUBY_VERSION >= '1.9'
require 'rubocop/rake_task'
RuboCop::RakeTask.new
end
PuppetLint.configuration.send('disable_80chars')
PuppetLint.configuration.relative = true
PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp']
desc 'Validate manifests, templates, and ruby files'
task :validate do
Dir['manifests/**/*.pp'].each do |manifest|
sh "puppet parser validate --noop #{manifest}"
end
Dir['spec/**/*.rb', 'lib/**/*.rb'].each do |ruby_file|
sh "ruby -c #{ruby_file}" unless ruby_file =~ %r{spec/fixtures}
end
Dir['templates/**/*.erb'].each do |template|
sh "erb -P -x -T '-' #{template} | ruby -c"
end
end
desc 'Run lint, validate, and spec tests.'
task :test do
[:lint, :validate, :spec].each do |test|
Rake::Task[test].invoke
end
end
modules/jupyter/examples/init.pp 0 → 100644
# The baseline for module testing used by Puppet Inc. is that each manifest
# should have a corresponding test manifest that declares that class or defined
# type.
#
# Tests are then run by using puppet apply --noop (to check for compilation
# errors and view a log of events) or by fully applying the test in a virtual
# environment (to compare the resulting system state to the desired state).
#
# Learn more about module testing here:
# https://docs.puppet.com/guides/tests_smoke.html
#
include ::jupyter
modules/jupyter/files/etc/config.py 0 → 100644
c.NotebookApp.allow_password_change = False
c.NotebookApp.allow_root = False
c.NotebookApp.ip = '*'
# change to 'localhost' to restrict access to the local machine
# c.NotebookApp.ip = 'localhost'
c.NotebookApp.port = 8888
c.NotebookApp.open_browser = False
c.NotebookApp.notebook_dir = '/home/jupyter'
modules/jupyter/files/etc/custom/custom.js 0 → 100644
define(['base/js/namespace'], function(Jupyter){
Jupyter._target = '_self';
});
modules/jupyter/files/jupyter.service 0 → 100644
[Unit]
Description = "Jupyer Notebook"
After = syslog.target
[Service]
Type = simple
Environment=JUPYTER_CONFIG_DIR=/etc/jupyter
ExecStart = jupyter notebook --config /etc/jupyter/config.py
StandardOutput = journal
StandardError = journal
Restart = always
RestartSec = 5
User = jupyter
[Install]
WantedBy = multi-user.target
modules/jupyter/files/set_password.py 0 → 100755
#!/usr/bin/env python3
import tempfile
import os
import click
import subprocess
from notebook.auth import passwd
@click.command()
@click.option("-c","--config",default="/etc/jupyter/config.py")
@click.password_option()
def main(config, password):
if not os.path.isfile(config):
print("Error config file [%s] does not exist, specify with -c" % config)
exit(1)
hashed_password = "c.NotebookApp.password = '%s'\n" % passwd(password)
config_lines = []
replaced_passwd = False
try:
with open(config, 'r') as cur_f:
for line in cur_f:
if "c.NotebookApp.password" in line:
line = hashed_password
replaced_passwd = True
config_lines += line
if not replaced_passwd:
print("Created the password configuration")
config_lines += hashed_password
else:
print("Updated the password configuration")
with open(config, 'w') as cur_f:
for line in config_lines:
cur_f.write(line)
except(PermissionError):
print("Cannot write to [%s], run with sudo" % config)
exit(1)
subprocess.run("service jupyter restart".split())
if __name__=="__main__":
main()
modules/jupyter/files/test.py 0 → 100644
c.NotebookApp.password = 'sha1:433faaee0d84:25c62af2f06422e1bfceafedcd772c762ab6b8db'
modules/jupyter/files/test_config.py 0 → 100644
c.NotebookApp.allow_password_change = False
c.NotebookApp.allow_root = False
c.NotebookApp.ip = 'localhost'
c.NotebookApp.port = 8888
c.NotebookApp.open_browser = False
c.NotebookApp.password = 'sha1:8978283e26f7:9833f53baed757b08aa471f43bd297583429461d'
modules/jupyter/manifests/config.pp 0 → 100644
class jupyter::config{
file{'/etc/jupyter':
ensure => directory,
recurse => true,
source => 'puppet:///modules/jupyter/etc',
owner => root,
group => root,
}
file{ '/etc/systemd/system/jupyter.service':
source => 'puppet:///modules/jupyter/jupyter.service',
owner => root,
group => root
}
file{ '/usr/local/bin/jupyter-set-password':
source => 'puppet:///modules/jupyter/set_password.py',
owner => root,
group => root,
mode => '0755'
}
exec { 'remove jupyter Desktop':
command => '/usr/bin/rm -rf /home/jupyter/Desktop',
}
}
modules/jupyter/manifests/init.pp 0 → 100644
class jupyter {
require joule
contain jupyter::install
contain jupyter::config
contain jupyter::service
Class['::jupyter::install']
-> Class['::jupyter::config']
~> Class['::jupyter::service']
}
modules/jupyter/manifests/install.pp 0 → 100644
class jupyter::install{
package {'jupyter':
ensure => present,
provider => pip3
}
user{'jupyter':
ensure => present,
managehome => true,
uid => 902,
}
}
modules/jupyter/manifests/service.pp 0 → 100644
class jupyter::service{
service {'jupyter':
enable => true,
ensure => running
}
}
modules/jupyter/metadata.json 0 → 100644
{
"name": "jdonnal-jupyter",
"version": "0.1.0",
"author": "jdonnal",
"summary": null,
"license": "Apache-2.0",
"source": "",
"project_page": null,
"issues_url": null,
"dependencies": [
{
"name": "puppetlabs-stdlib",
"version_requirement": ">= 1.0.0"
}
],
"data_provider": null
}
modules/jupyter/spec/classes/init_spec.rb 0 → 100644
require 'spec_helper'
describe 'jupyter' do
context 'with default values for all parameters' do
it { should contain_class('jupyter') }
end
end
modules/jupyter/spec/spec_helper.rb 0 → 100644
require 'puppetlabs_spec_helper/module_spec_helper'
modules/nginx/templates/nginx.conf.erb
...@@ -64,7 +64,7 @@ http { ...@@ -64,7 +64,7 @@ http {
include /etc/nginx/map.conf; include /etc/nginx/map.conf;
server{ server{
listen 80; listen 80;
set $subdomain_apps false; # set to true to enable subdomains (requires DNS) set $subdomain_apps false; # set to true to enable subdomains (requires server_name to be set as well)
include /etc/nginx/docs.conf; include /etc/nginx/docs.conf;
include /etc/nginx/lumen.conf; include /etc/nginx/lumen.conf;
<% if @nilmdb -%> <% if @nilmdb -%>
...@@ -73,7 +73,7 @@ http { ...@@ -73,7 +73,7 @@ http {
} }
server{ server{
listen 443 ssl; listen 443 ssl;
set $subdomain_apps false; # set to true to enable subdomains (requires DNS) set $subdomain_apps false; # set to true to enable subdomains (requires servern_name to be set as well)
include /etc/nginx/lumen.conf; include /etc/nginx/lumen.conf;
include /etc/nginx/joule.conf; include /etc/nginx/joule.conf;
<% if @nilmdb == true -%> <% if @nilmdb == true -%>
......
modules/rails_api/manifests/init.pp
...@@ -20,7 +20,7 @@ class rails_api { ...@@ -20,7 +20,7 @@ class rails_api {
vcsrepo{'/opt/api': vcsrepo{'/opt/api':
ensure => latest, ensure => latest,
provider => git, provider => git,
source => 'https://git.wattsworth.net/wattsworth/control-api.git', source => 'https://github.com/wattsworth/lumen-api.git',
require => User['rails'], require => User['rails'],
notify => [Exec['bundle_install','db_migrate'], File['/opt/api/config/database.yml']], notify => [Exec['bundle_install','db_migrate'], File['/opt/api/config/database.yml']],
owner => 'rails' owner => 'rails'
......
site.pp
...@@ -9,6 +9,7 @@ node 'default' { ...@@ -9,6 +9,7 @@ node 'default' {
include nginx include nginx
include docs include docs
include first_boot include first_boot
include jupyter
if($facts['image_type']=='nilm') { if($facts['image_type']=='nilm') {
include nilmdb include nilmdb
include nilm include nilm
......
vagrant/Vagrantfile
...@@ -23,7 +23,7 @@ Vagrant.configure("2") do |config| ...@@ -23,7 +23,7 @@ Vagrant.configure("2") do |config|
# within the machine from a port on the host machine. In the example below, # within the machine from a port on the host machine. In the example below,
# accessing "localhost:8080" will access port 80 on the guest machine. # accessing "localhost:8080" will access port 80 on the guest machine.
config.vm.network "forwarded_port", guest: 80, host: 8080 config.vm.network "forwarded_port", guest: 80, host: 8080
config.vm.network "forwarded_port", guest: 8080, host: 8081 config.vm.network "forwarded_port", guest: 8888, host: 8888
config.vm.network "forwarded_port", guest: 443, host: 4430 config.vm.network "forwarded_port", guest: 443, host: 4430
# Create a private network, which allows host-only access to the machine # Create a private network, which allows host-only access to the machine
# using a specific IP. # using a specific IP.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment