All-inclusive OSS Platform for Client/Server Applications
Status
- Rewrite in progress on
pl3
branch - Check Release for changes
- #platform:mozilla.org
Guidelines
Make it work, make it right, make it fast. -Kent Beck
The only way to go fast, is to go well. -Robert C. Martin
I built the platform
to enable smaller developers to rapidly create professional
software. The goal is to focus on the needs of business and customers, while being
able to rely on a stable technical foundation. This foundation is the platform
.
The platform
provides everything needed for developers to deliver professional,
business centric, web-based, client/server applications. It aims to be powerful
and relatively simple, self contained and not reliant on external services, so
the applications based on it can run on bare metal and in the cloud, on Linux and
Windows.
The platform
is not suited for every type of application and scale. It's designed
for small and medium sized business applications in the realm of digitalization,
automation and optimization of processes. For all business and customer facing components,
function and user experience have priority over a custom and fancy UI design.
The platform
allows developers to bootstrap a complete development
environment. This includes developer machines on Linux or Windows and a development
server that provides SCM,
CI/CD
capabilities and a repository manager.
While the platform
contains original pieces of software, it's standing on the
giant shoulders of Free/Libre and Open Source Software.
Bringing all these tools, libraries and frameworks together in a comprehensive form,
I hope the platform
can deliver real value for developers and is more than the sum
of its parts.
Architecture
The original pieces of software provided are the meta
and bootstrap
frameworks.
meta
is a full backend/frontend solution for building web-based client/server
applications in any domain, hence its name.
bootstrap
is a tool to build modules for installing and provisioning a system
with software.
util
is a collection basic Java/JavaScript building blocks used by all parts
of the platform
and in applications.
Util
Meta
- meta-core
- meta-client
- meta-engine
- Java based server, made up of a collection of services that can be expanded by applications with their own
- Provides WebSocket and REST API for clients
- meta-test
- meta-admin
- Web application to manage and monitor all functionality offered by
meta-engine
- Web application to manage and monitor all functionality offered by
- meta-admin-test
Bootstrap
- bootstrap-core
- Java based framework for building, provisioning and executing bootstrap modules on Linux and Windows
- bootstrap-core-test
- Bootstrap modules
- Bootstrap modules are based around simple JSON definitions that contain actions needed to install/configure software
- Bootstrap modules (eg.
bootstrap-dev-vm
for installing a developer machine) are in the top levelbootstrap
module - Custom variations and build/provision configs are in a seperate software
repository. In case of the platform
platform-bootstrap
- bootstrap-test
- Automated tests for provisioning bootstrap modules
Source repositories
Fossil SCM repositories:
- platform
- Monorepo containing all source code and bootstrap modules
- platform-bootstrap
- Configurations for
platform
specific bootstrap modules
- Configurations for
- platform-archetype Out of date / Obsolete with
pl3
- Archetype of a repository based on
platform
with sample application
- Archetype of a repository based on
- platform-archetype-bootstrap Out of date / Obsolete with
pl3
- Archetype of a bootstrap repository for customized bootstrap modules
If you just want to look at the source, download fossil binary and checkout the repositories:
fossil clone https://platform:platform@dev.rswk.ch/scm/platform
For a full walk-through, see the Setup How-To.
Objectives
- Automate what you can. Almost automated is better than nothing. Use automation as form of documentation
- Frequently update dependencies. Sometimes it's good to be on the bleeding edge
- Keep it simple, sometimes boring is better
- Minimize abstractions, dependencies and vendor lockin
- Remove cruft, YAGNI
- There are no solutions, only trade-offs
- Version numbers are a TL;DR for release notes
Code
- Use brackets for control flow statements
if (...) { return ...; } return ...;
- Use brackets for functions/lambdas
items.stream().filter((item) -> { // Java return ...; }).findFirst(); items.filter((item) => item.isNew()); // JavaScript items.filter(function (item) { // JavaScript multiple lines return ...; });
- Order attributes, fields, parameters, properties, etc. in ASCII order
public void login(final String password, final String user) ... enum ContactType { EMAIL, MAIL, PHONE } function (password, user) ... search({limit: 100, name: "Bob"}); <form data-bind="submit: register" id="register" novalidate>
- Don't use preincrement/postincrement operators
i = i + 1;
- Don't use negation
!
if (hasEnded == false) // Java if (hasEnded === false) // JavaScript if (Boolean(convertedValue) === false)
- Use
<
and<=
for comparisonsif (0 < limit && limit < 10) {}
- Use constants first in comparisons
if (MARKER.equals(currentMarker)) {}
- Use log format with comma separated properties
LOG.debug("message, property1={}, property2={}", property1, property2); // Java LOG.debug(`message, property1=${property1}, property2=${property2}`); // JavaScript
- Lists should be output in format
property3=[el1,el2,el3]
- Lists should be output in format
- Comments
- Put effort into expressive code and use descriptive variable and method names
- Comments are a last resort
- Comments lie
- Use
// FIXME
and// TODO
Formatting
Setup according to dev-vm
guide and
use IntelliJ formatter and common sense/existing code.
Documentation
- Maven Site
- Contents of
src/site
- Explanations, how-to guides and tutorials
- Boostrap module reference documentation in
bootstrap.md
- Contents of
- JavaDoc
- Reference documentation for Java code
- Packages with
package-info.java
- Use for overview, architecture, etc.
- Java files according to JavaDoc standards
- JSDoc
Build
- All artifacts are built with Maven, or by Maven triggering NPM for JavaScript modules
- Java JAR artifacts
- JAR is based on a single Maven module
- Code is structured into Java packages in
src/main/java
- Module dependencies are handled by Maven in
pom.xml
- Dependency convergence is not allowed
- Source is compiled and packaged as JAR/Shaded JAR
- A source JAR is also generated and installed to the Maven repository
- Test modules are regular Maven modules and seperated from the modules under test
- JAR files are published to hosted Maven repository on Nexus server
- JavaScript
- Code is structured into ES6 modules
in
src/main/js/modules
with additional subdirectoriescss
for CSSstatic
for audio, images and other static files
- Dependencies to other NPM packages are configured in
src/main/resources/js/package.json
- NPM packages are managed in a workspace using the top level
package.json
- Shell scripts in
util-web
, used in Maven build profilesweb-install
- Run
npm install
- Run
web-build
- Run
npm run build
- Run
web-publish
- Run
npm publish
- Run
web-doc
- Run
npm run doc
- Run
- ZIP web application artifacts
- Requires
web-build.js
script- Call
esbuild
to bundle resources
- Call
- Static files are copied using
maven-resources-plugin:copy-resources
- Published to hosted Maven repository on Nexus server using
maven-assembly-plugin
withweb-dist.xml
assembly fromutil-web
- During development
- Run
npm run build -- watch
to start esbuildwatch
- Run
mvn resources:copy-resources@web-build-resources
to copy static files
- Run
- Requires
- NPM package artifacts
- Published to hosted NPM registry on Nexus server using
web-publish
- Published to hosted NPM registry on Nexus server using
- Code is structured into ES6 modules
in
- Bootstrap ZIP artifacts
- ZIP is based on Maven module
- Bootstrap resources are in
src/main/resources/bootstrap
- Some standard directories are expected, but all directories and files
are packaged into the ZIP
config
contains server and client configuration. These are usually files that contain variables (eg.{serverDomain}
) that are replaced by specific values frombootstrap.json
or environment variables during executiondefinitions
contains bootstrap definition JSON files that describe the installation processfixture
contains static files that are used as part of the installation, eg. shell scripts, static resources, etc.
- Some standard directories are expected, but all directories and files
are packaged into the ZIP
- Dependencies to other bootstrap module are resolved using Maven in
pom.xml
. The contents of these bootstrap modules become part of the final ZIP, so be aware of file name clashes. All files from these included modules can be referenced like they are part of the current bootstrap module - Dependencies on JAR and ZIP binary artifacts (or any other Maven artifact)
are resolved by using
maven-dependency-plugin
. The articacts are copied into thebin
directory in the final ZIP
- Documentation Maven Site
- Site is configured/enabled on the root Maven module
- Generated using
maven-site-plugin
- Configuration in
reporting
section ofpom.xml
- Site generation for sub modules is disabled using
<maven.site.skip>true</maven.site.skip>
- Generated using
- Aggregated JavaDoc is generated using
maven-javadoc-plugin
- JSDoc is generated by
web-doc
shell script in individual modules and written tosrc/site/resources/jsdoc
of root Maven module
- Site is configured/enabled on the root Maven module
Test
- In general
- Unit tests follow the pattern of
/* Given */ /* When */ /* Then */
- Unit tests follow the pattern of
- JUnit Java tests
- Unit, integration tests for Java code
- Test classes are in a
test
package- Eg. for
ch.rswk.util.core
inch.rswk.util.core.test
- Eg. for
- Test resources are in
src/main/resources/fixture
- Tests are run during build using
maven-surefire-plugin
/maven-failsafe-plugin
- QUnit JavaScript tests
- Unit tests for JavaScript code
- Test suite is in
src/main/js/qunit.html
and test modules are imported and run insrc/main/js/modules/tests.js
- Test module is named after the module under test, eg.
src/main/js/modules/client-test.js
forclient
module - Tests are run in WebDriver UI tests, see below
- WebDriver UI tests
- Selenium WebDriver based tests in Java
WebDriverTest
classes are intest
package and extend AbstractWebDriver- Tests run as part of JUnit suite
- Bootstrap provisioning tests
BootstrapTest
classes are intest
package and extend AbstractBootstrapTest- Tests are not run as part of CI builds, but used as a manual tool to test bootstrap modules
- Each test case
- Builds bootstrap module for target environment
- Creates and provisions new server instance with module
- Deletes server instance when done
Commit
Checklist:
- Tests green?
- Tests for new code?
- Changed configurations?
- Adapt bootstrap configurations
- Documentation updated?
- Release notes updated?
Commit message format:
Most important thing I did in this commit, a good summary of my changes.
- List of things that changed
- More stuff
Release
platform
- This also applies to repositories based on the
platform
, likeplatform-archetype
- SCM branches
trunk
- Development happens here
- Always compiles
- Tests are green
stable
- Manually merge/graft commits from
trunk
- No development, just minor commits (release notes, fixes)
- Manually merge/graft commits from
- Creating more branches is always an option
- CI jobs
platform-snapshot
- Build, test and deploy contents of
trunk
to Nexus
- Build, test and deploy contents of
platform-release
- Build, test and deploy contents of
stable
to Nexus - Manual steps are required in preparation of and after a release, see Release document
- Build, test and deploy contents of
- This also applies to repositories based on the
platform-bootstrap
- This also applies to bootstrap repositories based on the
platform
, likeplatform-archetype-bootstrap
- SCM branches
trunk
- Root directory per customer, eg.
customerone
- One root directory for the organization itself, eg.
mycompany
for infrastructure and development bootstrap modules - Root directories contain one directory per bootstrap module, eg.
bootstrap-myapp-server
- Bootstrap module directory contains the Jenkins job configuration, build/provision JSON configs and optional customizing files
- Root directory per customer, eg.
- CI jobs
- One per bootstrap module, eg.
bootstrap-myapp-server
- Parameters for build (update/install), customer, etc.
- One per bootstrap module, eg.
- This also applies to bootstrap repositories based on the
Versioning
platform
versioning is based on Semantic versioning. Major is not expected to
change until the current rewrite. When Minor changes, it's a good idea to check
the release notes. Changes in Patch means internal changes, fixes or new features
that should be backwards compatible.
Consider using a versioning scheme that fits your development style:
Setup How-To
Requirements:
dev-vm
developer machine- Ubuntu 22.10 or Windows 11
- 4+ CPU cores
- 4+ GB RAM
- 20 GB disk space
dev-server
development server- Ubuntu 22.04.01
- 4 CPU cores
- 8 GB RAM
- 100+ GB disk space (depending on number of stored/archived artifacts)
Bootstrap dev-vm
Setup and bootstrap VM
Setup a fresh VM, eg. using VirtualBox, using one of the ISOs:
- Ubuntu 23.10
- Windows 11
- Download Windows 11 Disk Image (ISO)
- Installation
I don't have a product key
>Windows 11 Pro
- Setup for work
Sign-in options
>Domain join
Once the OS installation is finished, use the following scripts to start the bootstrap process.
- Ubuntu terminal
- Download bootstrap module
cd /var/tmp wget -O bootstrap.zip 'https://rswk.ch/bootstrap-dev-vm-nix.zip'
- Unzip module
sudo apt install -y unzip unzip bootstrap.zip -d bootstrap cd bootstrap chmod +x bootstrap.sh
- Execute module
sudo ./bootstrap.sh
- Download bootstrap module
- Windows PowerShell/Terminal as Admin
- Download bootstrap module
mkdir C:\tmp cd C:\tmp Set-Variable ProgressPreference SilentlyContinue Invoke-WebRequest -Uri 'https://rswk.ch/bootstrap-dev-vm-win.zip' -OutFile bootstrap.zip
- Unzip module
Expand-Archive -LiteralPath bootstrap.zip -DestinationPath bootstrap cd bootstrap
- Execute module
Set-ExecutionPolicy Unrestricted .\bootstrap.ps1
- Download bootstrap module
- In case of errors, check
bootstrap.log
in thebootstrap
directoryERROR restart point created. see log for errors, fix errors, then run bootstrap script again to resume execution
- Wait for bootstrap process to finish
executeModule done, elapsed=...
Continue with the manual steps to complete the setup.
Explore platform-archetype
Before we dive into the example application, let's make sure everything is setup
correctly by running the meta-admin
web application:
- In IDEA, look for module
meta-admin-test
inframework/meta
- Open
WebDriverTest
class- In
index
method, add breakpoint on line first of line afterrunUiTest
- Make sure break point suspend option is set to
Thread
- In
- Click on run symbol for
index
method and selectDebug index()
- Two browser windows should open, with Chromium opening the
meta-admin
client at https://localhost.platform.rswk.ch:9090/meta-admin/src/main/resources/ - Login with user
plat
and passwordplat
TODO do something in admin?
- Open
- Abort the test case
With that done, we can explore the archetype myapp
application. It's part of
platform-archetyp
, the reference for a repository based on platform
.
- Run a full Maven build of
platform-archetype
withmvn -DskipTests
- Linux
~/scm/platform-archetype
- Windows
C:\scm\platform-archetype
- Linux
- In IDEA, use File > Open and select the directory we used above
- Note that we have to repeat most of the project setup steps in
Project Structure
andSettings
, like we did for theplatform
repository
- Note that we have to repeat most of the project setup steps in
- Explore
myapp-core
,myapp-engine
,myapp-test
modules inapplication/myapp
- In Visual Studio Code
- Explore
platform-archetype/application/myapp/myapp-web/src/main/resources
for the client application
- Explore
- Now everything is ready to try, extend and test the
myapp
application - Debug
WebDriverTest#index
like we did inmeta-admin-test
- Login to the client application using same login as we did for
meta-admin
- Open the client in a second tab, then click on one of the tiles and the clients should synchronize
TODO explain how things work together. model, backend service, client, tests, etc.
In addition to platform-archetyp
, there is also a reference bootstrap source
repository in platform-archetype-bootstrap
containing bootstrap module customizations.
Bootstrap watchdog-server
- Watchdog is a central
meta-engine
instance withmeta-admin
client that is used for monitoring and maintenance for othermeta-engine
based servers in an organization - It is not required for other engines to work, but it is required for automated
provisioning of bootstrap modules, either for testing with
BootstrapTest
or for Jenkins jobs that deploy bootstrap modules - Setup Ubuntu VM, eg. with Vultr or any cloud computing provider
- Setup a DNS A record for domain you control that points to the IP of the VM
- Vultr offers a free DNS service
- Example domain used for watchdog server is
watchdog.mycompany.ch
- Replace all instances in
platform-archetype-bootstrap
of this with a domain you control and would like to use
- Replace all instances in
- Build
bootstrap-watchdog-server
module in IDEA- In
platform
project underframework/bootstrap/bootstrap-core
, right click onMain.java
> Modify Run Configuration - Name
Main Watchdog
- Program arguments (On Linux, use
~/scm/
instead ofC:/scm
)-build C:/scm/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/build.json -customize C:/scm/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server -merge C:/scm/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/bootstrap_install.json -module C:/scm/platform/bootstrap/bootstrap-watchdog-server
- Environment variables for variables in
bootstrap.json
EMAIL_PASSWORD=my_email_password; WATCHDOG_PW=my_watchdog_user_pw;
- In
Upload and bootstrap watchdog-server
- Upload
platform/bootstrap/bootstrap-watchdog-server/target/bootstrap-watchdog-server-VERSION.zip
to server's/var/tmp
directory - Unzip module
sudo apt install -y unzip find /var/tmp -name 'bootstrap-*.zip' -exec unzip {} -d /var/tmp/bootstrap \; cd /var/tmp/bootstrap chmod +x bootstrap.sh
- Execute module
sudo ./bootstrap.sh
meta-admin
should be available athttps://watchdog.mycompany.ch
. Login with userwatchdog
and passwordmywatchdogpw
- To have a different user or additional users for the other engines to use, create
fixture.json
inplatform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/config/engine
to override the default fixtures inplatform/bootstrap/bootstrap-watchdog-server/src/main/resources/bootstrap/config/engine/fixture.json
Bootstrap myapp-server
- Let's try automated provisioning with
BootstrapTest
using the watchdog server we just setup BootstrapTest
(which usesProvision
class under the hood) uses cloud computing provider Vultr to create and deploy VMs. See Vultr Cheatsheet- In IDEA, open
BootstrapTest
inplatform-archetype/bootstrap/bootstrap-test
- Remove
@Disabled
annotation - This test uses
localhost.platform.rswk.ch
domain in configuration files. To access the server after installation, open your localhosts
file and temporarily change the entry forlocalhost.platform.rswk.ch
to the IP of the deployed VM- Linux
/etc/hosts
- Windows
C:/Windows/System32/drivers/etc/hosts
- Linux
- Test can also run with real domains
- Setup DNS A record for a reserved IP with Vultr
- Set
keystoreFile
inpropecrties
map toAbstractBootstrapTest.CERTBOT_KEYSTORE_FILE
- Set
serverDomain
inproperties
map, eg. totest.mycompany.ch
- Pass
certbot
flag tonewProvisionTestExecuteConfig
method - Set
provisiontest_vultr_reservedip
to the reserved IP ID
- Set
- Setup DNS A record for a reserved IP with Vultr
- Remove
- Right click on
BootstrapTest.java
> Modify Run Configuration- Name
BootstrapTest MyAppServer
- Environment variables
provisiontest_vultr_apikey=XXX; provisiontest_vultr_firewall=XXX; provisiontest_vultr_reservedip=; provisiontest_vultr_serverpassword=Hu3m4?mFKk2S9!=; provisiontest_vultr_sshkey=XXX; provisiontest_watchdog_engine=watchdog.mycompany.ch; provisiontest_watchdog_password=my_watchdog_user_pw; provisiontest_watchdog_user=watchdog;
- All
XXX
above need to be replaced with actual values from your Vultr settings. See Vultr Cheatsheet
- Name
- Set a breakpoint in
myAppServer
onassertWatchdogStatus
call - When breakpoint is reached, the server should be up and running and the client
accessible at
https://localhost.platform.rswk.ch
(or the domain you specified)- Check your watchdog server. It should have received the installation status of the new server with a log of the bootstrap process
- Resume the debugging session and the test resources are cleaned up. If everything went as expected, the test should continue to provision the next target environment
Bootstrap dev-server
For professional software development, it's recommended to setup a development server where all the source repositories, automated CI/CD builds and binary artifacts can be managed/stored.
- Setup Ubuntu VM, eg. with Vultr, another cloud computing provider, or even a local hypervisor software if you want to run the server on premise
- Setup a DNS A record for domain you control that points to the IP of the VM
- Example domain used for development server is
dev.mycompany.ch
- Replace all instances of
dev.mycompany.ch
inplatform-archetype-bootstrap
with the domain you control and would like to use
- Replace all instances of
- To just test the setup of a development server, you can run
BootstrapTest#devServer
inplatform/bootstrap/bootstrap-test
, like we did withmyapp-server
- Build
bootstrap-dev-server
module in IDEA- In
platform
project underframework/bootstrap/bootstrap-core
, right click onMain.java
> Modify Run Configuration - Name
Main DevServer
- Program arguments
-build C:/scm/platform-archetype-bootstrap/mycompany/bootstrap-dev-server/build.json -customize C:/scm/platform-archetype-bootstrap/mycompany/bootstrap-dev-server -module C:/scm/platform/bootstrap/bootstrap-dev-server
- Environment variables used in
boostrap.json
NEXUS_PW=my_nexus_admin_password;
- In
Upload and bootstrap dev-server
- Upload
platform/bootstrap/bootstrap-dev-server/target/bootstrap-dev-server-VERSION.zip
to server's/var/tmp
directory - Unzip module
sudo apt install -y unzip find /var/tmp -name 'bootstrap-*.zip' -exec unzip {} -d /var/tmp/bootstrap \; cd /var/tmp/bootstrap chmod +x bootstrap.sh
- Execute module
sudo ./bootstrap.sh
Continue with the manual steps to complete the setup.
Turtles all the way down
- At this point we have setup
- Developer machine
- Development server
- Watchdog server
- But all of this was done from the local developer machine. With the development server, we can now store and build everything from a central place and get other people setup as well
- Clone the
mycompany
andmycompany-boostrap
repositories to your local machine- Copy contents of
platform-archetype
tomycompany
andplatform-archetype-bootstrap
tomycompany-boostrap
- Rename artifacts, packages, etc. to match your organization
- Commit changes to server
- Copy contents of
- Adapt CI jobs in Jenkins to match your organization
- Repository and job names
bootstrap-dev-vm
- Build bootstrap module for developer machines based on the customizing
in
platform-archetype-bootstrap/mycompany/bootstrap-dev-vm
- Build bootstrap module for developer machines based on the customizing
in
bootstrap-dev-server
- Build bootstrap module for development server based on the customizing
in
platform-archetype-bootstrap/mycompany/bootstrap-dev-server
- Build bootstrap module for development server based on the customizing
in
bootstrap-watchdog-server
- Build bootstrap module for watchdog server with customizing in
platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server
- Set
BUILD
param toinstall
to build the module. Must be manually installed on a new watchdog server, since provisioning depends on a watchdog server - Set
BUILD
param toupdate
to build the module and deploy it to existing watchdog server
- Set
- Build bootstrap module for watchdog server with customizing in
bootstrap-myapp-server
- Example to build and provision a customer
myapp
server.- Set
CUSTOMER
param tocustomerone
to build job with customizing inplatform-archetype-bootstrap/customerone/bootstrap-myapp-server
- Set
BUILD
param toinstall
to provision a new server using Vultr - Set
BUILD
param toupdate
to update an existing server to a new version
- Set
- Example to build and provision a customer
bootstrap-myapp-cd
- Example for continuous delivery job. Updates all
myapp
servers that run compatible, old version to a new release version
- Example for continuous delivery job. Updates all
Cheatsheet
Keystores
- Java keystore list certificates
keytool -v -list -keystore keystore.pfx
- Java keystore import existing keystore (see below how to create PFX from PEM files)
keytool -importkeystore -deststorepass 123 -destkeystore keystore.pfx -srckeystore tmpkeystore.pfx -srcstoretype PKCS12
- Java keystore remove certificate
keytool -delete -alias server -keystore keystore.pfx
- Install OpenSSL, eg. see bootstrap definition
platform/bootstrap/bootstrap-common/src/main/resources/bootstrap/definitions/openssl.json
- Configuration
set RANDFILE=C:\data\openssl.rnd set OPENSSL_CONF=C:\tool\httpd\conf\openssl.cnf
- Convert PEM certificate and key to Java keystore
openssl pkcs12 -export -in server.crt -inkey server.key -out tmpkeystore.pfx -name server -password pass:123
- Export PEM certificates from Java keystore
openssl pkcs12 -in keystore.pfx -out keystore.pem -nodes
- Self signed CA and server certificate with SAN
- Adapt configuration in
openssl.cnf
[ req ] req_extensions = v3_req [ v3_req ] basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment subjectAltName = @alt_names [ alt_names ] DNS.1 = myapp.com [ v3_ca ] authorityKeyIdentifier = keyid,issuer basicConstraints = critical,CA:true,pathlen:3 keyUsage = critical, cRLSign, keyCertSign nsCertType = sslCA, emailCA subjectKeyIdentifier = hash
- Generate certificate
openssl genrsa -out ca.key 2048 openssl req -new -x509 -extensions v3_ca -days 720 -key ca.key -sha256 -out ca.crt openssl genrsa -out server.key 2048 openssl req -extensions v3_req -sha256 -new -key server.key -out server.csr openssl x509 -req -extensions v3_req -days 720 -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -extfile C:\tool\httpd\conf\openssl.cnf
- Adapt configuration in
- Links
Java
- Create runtime from JDK
- Download JDK
- Generate list of all
.jmod
files injmod
directory- Eg. with
ls
orjava --list-modules
- Eg. with
- Run jlink
- Start shell in
bin
directory - Replace
XXX
below with comma separated list of all modules (without.jmod
) - Linux
./jlink --module-path ../jmods --add-modules "XXX" --output /var/lib/platform/tool/java17
- Windows
.\jlink.exe --module-path ..\ --add-modules "XXX" --output C:\tool\java17
- Start shell in
- Create 7z archive from content of
java17
directory- Linux
java17-nix.7z
- Windows
java17-win.7z
- Linux
- Upload to
platform
dev server
- Zulu Mission Control
- Open
zmc.ini
and add path to Java as first argument- Linux
-vm /var/lib/platform/tool/jdk17/bin
- Windows
-vm C:/tool/jdk17/bin/server/jvm.dll
- Linux
- Open
Links
- https://anthonyhobday.com/sideprojects/saferules/
- https://app.diagrams.net/
- https://basecamp.com/shapeup
- https://excalidraw.com/
- https://exiftool.org/
- https://freesound.org/
- https://input.fontbureau.com/download/
- https://regex101.com/
- https://regexper.com
- https://www.shellcheck.net/
- https://www.vultr.com/resources/developers/
Logging
- Java
- Tests in
platform
framework/util/util-core-test/src/main/resources/logback-test.xml
- Tests in Engine modules
myapp-test/src/main/resources/logback-test.xml
- In general
- Add a
logback.xml
(orlogback-test.xml
) tosrc/main/resources
- Add a
- Runtime
- For
meta-engine
inengine.json
to overwrite the level of a logger"logging": { "loggers": { "ch.rswk.meta.engine": "DEBUG" } }
- For
- Tests in
- JavaScript based on
util-web
- In
config.js
setdebug: true
- QUnit tests
Logger.useDefaults({ defaultLevel: Logger.DEBUG });
- Clients using
meta-client/logging
withmeta-admin
- Notifications > Send client notification
- Any channel the client is subscribed to
- Action
INFO
- Data
{"logLevel": "DEBUG"}
- Notifications > Send client notification
- In
Lucene
- Inspect index with Luke
- Download latest Lucene distribution
- Extract ZIP and set
JAVA_HOME
in Luke shell scripts- Linux
luke/luke.sh
set$JAVA_HOME/bin/java
- Windows
luke\luke.bat
set%JAVA_HOME%\bin\java
- Linux
- Run shell script
- Re-index existing index, eg. after new Lucene version
- Use
clone
task fromEngineUpdater
- Using
engine-clone.json
from a bootstrap module - Using IDEA JUnit
MainTest.update
test method with changednewMetaServiceTestConfig
(see below)
- Using
- Use
- Re-index existing index, eg. after changes to meta model
- Implement
MetaUpdater
/MetaUserUpdater
to migrate existing entities with theclone
task - See
EngineUpdaterSchemaTest.java
- Implement
- Test with existing index
- Get copy of index you would like to run locally
- Manually create a
7z
-file from an existingwork
directory - Or backup and download index with
meta-admin
>File
>Search backups
- Manually create a
- Upload in local
meta-admin
>File
- Enter correct tags, eg. for
MetaService
:backup,ch.rswk.meta.engine.MetaService
- Enter correct tags, eg. for
- Use restore function on uploaded file
- Get copy of index you would like to run locally
- Alternative way to test with existing index
- Extract/copy index to a local directory, eg.
C:/tmp/index/work
- In
MetaServiceTest.newMetaServiceTestConfig
, setluceneIndexDir
to parent directory, eg.C:/tmp/index
- When running
MetaServiceTest
, make sure to disablecleanup
inteardown
- Extract/copy index to a local directory, eg.
Maven
- Default goals for
platform
parent are set toclean install
- Only build current module and not full reactor
mvn -N
- Run specific test
mvn -DfailIfNoTests=false -Dtest=MyTest
mvn -DfailIfNoTests=false -Dtest=ch.rswk.meta.core.*.*Test
- Skip tests
mvn -DskipTests
- Only build certain modules
mvn -pl meta-core,meta-client
- Only build certain module, including dependencies
mvn -pl meta-engine -am
- List module dependency tree
mvn dependency:tree
- Upload third party artifact to Nexus
mvn deploy:deploy-file -DgroupId='ch.othercompany' -DartifactId=theframework -Dversion='1.0.0' -Dpackaging=jar -Dfile='theframework.jar' \ -DgeneratePom=true -DrepositoryId=dev-server -Durl=https://dev.mycompany.ch/nexus/repository/thirdparty/
- Use
-DpomFile='theframework.pom'
instead of-DgeneratePom=true
to provide your own POM
- Use
Vultr
- Account
- API
- Enable API and store key somewhere safe
- Used in provision config
- Add IP/subnets that are going to call the API to access control (dev machines, dev servers, etc.)
- Enable API and store key somewhere safe
- SSH Keys
- Generate new keys
- Linux
ssh-keygen -f ~/.ssh/vultr_rsa
cat ~/.ssh/vultr_rsa.pub
- Windows
- Bitvise SSH Client > Client key manager > Generate new
- Enter passphrase > Generate
- Export > Export public key > OpenSSH format
- Linux
- Add SSH Key
- Copy
id
query parameter from URL and store somewhere safe- Used in provision config
- Generate new keys
- API
- Products > Firewall > Add Firewall Gruop
- Accept SSH from your IP
- Accept MS RDP from your IP
- Accept HTTP, HTTPS from anywhere
- Copy
Group ID
and store somewhere safe- Used in provision config
- Products > Network > Reserved IPs
- Add for domains with fixed DNS A records and store
- Copy
id
query parameter from URL and store somewhere safe- Used in provision config
Web
- Development
- In
WebDriverTest
, put breakpoint on first line afterrunUiTest(...)
. In IDEA, make sure to set breakpointSuspend
toThread
- Use Chromium/Firefox instance opened by the
WebDriverTest
for client development - If you need to access any service
runtime.getServices(MockGatewayService.class).get(0)
- In
- Service workers
- Force reload and refresh without using dev tools
- In
config.js
/config.json
change version - In
worker-init.js
change version in comment on first line
- In
- Force reload and refresh without using dev tools
- URL for tests with
ch.rswk.util.web.test.WebServer
https://localhost.platform.rswk.ch:8088/qunit.html
- URL for tests with
meta-engine
andHttpService
https://localhost.platform.rswk.ch:9090/
- Execute action via REST-like API from shell
- Linux Bash
#!/bin/bash EXECUTE_REQUEST='{"actionId":"myaction","param":{},"reqt":"ecq"}' RESPONSE=$(curl -s --fail --header "Accept:application/json" \ --header "Authorization:Bearer $(curl https://myapp.mycompany.ch/rs/token -s --data "user=myuser" --data "password=pw123")" \ --header "Content-Type:application/json" --data "$EXECUTE_REQUEST" \ --request POST https://myapp.mycompany.ch/rs/execute) if [ $? -eq 22 ]; then echo "Error" else echo "$RESPONSE" fi
- Windows PowerShell/Terminal
$executeRequest = @{ actionId = 'myaction' param = @{} reqt = 'ecq' } $headers = @{ 'Accept' = 'application/json' 'Authorization' = $("Bearer " + (Invoke-WebRequest -Body @{user='myuser';password='pw123'} -Method 'POST' -Uri https://myapp.mycompany.ch/rs/token)) 'Content-Type' = 'application/json' } try { $response = Invoke-WebRequest -Body ($executeRequest | ConvertTo-Json) -Headers $headers -Method 'POST' -Uri https://myapp.mycompany.ch/rs/execute echo $response } catch { $err = $_.Exception echo $err.Response }
- Linux Bash
Software references
Disclaimer
- Highcharts is only free for non-commercial use. If you plan to use it in a commercial setting, please buy a license
- SMS delivery in GatewayService depends on an external provider based in Switzerland: iNetWorx SMS-Gateway. Please contact them to order a SMS package. No affiliation with iNetWorx, just happy customers for many years
- Automated provisioning of bootstrap modules depends on having an account with Vultr. We would like to add more providers in the future, but are happy right now with the simplicity, functionality and pricing provided. Again, no affiliation with Vultr