All-inclusive OSS Platform for Client/Server Applications
Status
Guidelines
Make it work, make it right, make it fast. -Kent Beck
The only way to go fast, is to go well. -Robert C. Martin
I built the platform to enable smaller developers to rapidly create professional
software. The goal is to focus on the needs of business and customers, while being
able to rely on a stable technical foundation. This foundation is the platform.
The platform provides everything needed for developers to deliver professional,
business centric, web-based, client/server applications. It aims to be powerful
and relatively simple, self contained and not reliant on external services, so
the applications based on it can run on bare metal and in the cloud, on Linux and
Windows.
The platform is not suited for every type of application and scale. It's designed
for small and medium sized business applications in the realm of digitalization,
automation and optimization of processes. For all business and customer facing components,
function and user experience have priority over a custom and fancy UI design.
The platform allows developers to bootstrap a complete development
environment. This includes developer machines on Linux or Windows and a development
server that provides SCM,
CI/CD
capabilities and a repository manager.
While the platform contains original pieces of software, it's standing on the
giant shoulders of Free/Libre and Open Source Software.
Bringing all these tools, libraries and frameworks together in a comprehensive form,
I hope the platform can deliver real value for developers and is more than the sum
of its parts.
Architecture
The original pieces of software provided are the meta and bootstrap frameworks.
meta is a full backend/frontend solution for building web-based client/server
applications in any domain, hence its name.
bootstrap is a tool to build modules for installing and provisioning a system
with software.
util is a collection basic Java/JavaScript building blocks used by all parts
of the platform and in applications.
Util
Meta
- meta-core
- meta-client
- meta-engine
- Java based server, made up of a collection of services that can be expanded by applications with their own
- Provides WebSocket and REST API for clients
- meta-test
- meta-admin
- Web application to manage and monitor all functionality offered by
meta-engine
- Web application to manage and monitor all functionality offered by
- meta-admin-test
Bootstrap
- bootstrap-core
- Java based framework for building, provisioning and executing bootstrap modules on Linux and Windows
- bootstrap-core-test
- Bootstrap modules
- Bootstrap modules are based around simple JSON definitions that contain actions needed to install/configure software
- Bootstrap modules (eg.
bootstrap-dev-vmfor installing a developer machine) are in the top levelbootstrapmodule - Custom variations and build/provision configs are in a seperate software
repository. In case of the platform
platform-boostrap(Already removed as part of thepl3rewrite)
- bootstrap-test
- Automated tests for provisioning bootstrap modules
Source repositories
Fossil repositories:
platform- Monorepo containing all source code and bootstrap modules
platform-bootstrap(Already removed as part of thepl3rewrite)- Configurations for
platformspecific bootstrap modules
- Configurations for
If you just want to look at the source, download fossil binary and checkout the repositories:
fossil clone https://platform:platform@dev.rswk.ch/fossil/platform
For a full walk-through, see the Setup How-To.
Objectives
- Automate what you can. Almost automated is better than nothing. Use automation as form of documentation
- Frequently update dependencies. Sometimes it's good to be on the bleeding edge
- Keep it simple, sometimes boring is better
- Minimize abstractions, dependencies and vendor lockin
- Remove cruft, YAGNI
- There are no solutions, only trade-offs
- Version numbers are a TL;DR for release notes
Code
- Use brackets for control flow statements
if (...) { return ...; } return ...; - Use brackets for functions/lambdas
items.stream().filter((item) -> { // Java return ...; }).findFirst(); items.filter((item) => item.isNew()); // JavaScript items.filter(function (item) { // JavaScript multiple lines return ...; }); - Order attributes, fields, parameters, properties, etc. in ASCII order
public void login(final String password, final String user) ... enum ContactType { EMAIL, MAIL, PHONE } function (password, user) ... search({limit: 100, name: "Bob"}); <form data-bind="submit: register" id="register" novalidate> - Don't use preincrement/postincrement operators
i = i + 1; - Don't use negation
!if (hasEnded == false) // Java if (hasEnded === false) // JavaScript if (Boolean(convertedValue) === false) - Use
<and<=for comparisonsif (0 < limit && limit < 10) {} - Use constants first in comparisons
if (MARKER.equals(currentMarker)) {} - Use log format with comma separated properties
LOG.debug("message, property1={}, property2={}", property1, property2); // Java LOG.debug(`message, property1=${property1}, property2=${property2}`); // JavaScript- Lists should be output in format
property3=[el1,el2,el3]
- Lists should be output in format
- Comments
- Put effort into expressive code and use descriptive variable and method names
- Comments are a last resort
- Comments lie
- Use
// FIXMEand// TODO
Formatting
Setup according to dev-vm guide and
use IntelliJ formatter and common sense/existing code.
Documentation
- Maven Site
- Contents of
src/site- Explanations, how-to guides and tutorials
- Boostrap module reference documentation in
bootstrap.md
- Contents of
- JavaDoc
- Reference documentation for Java code
- Packages with
package-info.java- Use for overview, architecture, etc.
- Java files according to JavaDoc standards
- JSDoc
Build
- All artifacts are built with Maven, or by Maven triggering NPM for JavaScript modules
- Java JAR artifacts
- JAR is based on a single Maven module
- Code is structured into Java packages in
src/main/java - Module dependencies are handled by Maven in
pom.xml- Dependency convergence is not allowed
- Source is compiled and packaged as JAR/Shaded JAR
- A source JAR is also generated and installed to the Maven repository
- Test modules are regular Maven modules and seperated from the modules under test
- JAR files are published to hosted Maven repository on Nexus server
- JavaScript
- Code is structured into ES6 modules
in
src/main/js/moduleswith additional subdirectoriescssfor CSSstaticfor audio, images and other static files
- Dependencies to other NPM packages are configured in
src/main/resources/js/package.json - NPM packages are managed in a workspace using the top level
package.json - Shell scripts in
util-web, used in Maven build profilesweb-install- Run
npm install
- Run
web-build- Run
npm run build
- Run
web-publish- Run
npm publish
- Run
web-doc- Run
npm run doc
- Run
- ZIP web application artifacts
- Requires
web-build.jsscript- Call
esbuildto bundle resources
- Call
- Static files are copied using
maven-resources-plugin:copy-resources - Published to hosted Maven repository on Nexus server using
maven-assembly-pluginwithweb-dist.xmlassembly fromutil-web - During development
- Run
npm run build -- watchto start esbuildwatch - Run
mvn resources:copy-resources@web-build-resourcesto copy static files
- Run
- Requires
- NPM package artifacts
- Published to hosted NPM registry on Nexus server using
web-publish
- Published to hosted NPM registry on Nexus server using
- Code is structured into ES6 modules
in
- Bootstrap ZIP artifacts
- ZIP is based on Maven module
- Bootstrap resources are in
src/main/resources/bootstrap- Some standard directories are expected, but all directories and files
are packaged into the ZIP
configcontains server and client configuration. These are usually files that contain variables (eg.{serverDomain}) that are replaced by specific values frombootstrap.jsonor environment variables during executiondefinitionscontains bootstrap definition JSON files that describe the installation processfixturecontains static files that are used as part of the installation, eg. shell scripts, static resources, etc.
- Some standard directories are expected, but all directories and files
are packaged into the ZIP
- Dependencies to other bootstrap module are resolved using Maven in
pom.xml. The contents of these bootstrap modules become part of the final ZIP, so be aware of file name clashes. All files from these included modules can be referenced like they are part of the current bootstrap module - Dependencies on JAR and ZIP binary artifacts (or any other Maven artifact)
are resolved by using
maven-dependency-plugin. The articacts are copied into thebindirectory in the final ZIP
- Documentation Maven Site
- Site is configured/enabled on the root Maven module
- Generated using
maven-site-plugin - Configuration in
reportingsection ofpom.xml - Site generation for sub modules is disabled using
<maven.site.skip>true</maven.site.skip>
- Generated using
- Aggregated JavaDoc is generated using
maven-javadoc-plugin - JSDoc is generated by
web-docshell script in individual modules and written tosrc/site/resources/jsdocof root Maven module
- Site is configured/enabled on the root Maven module
Test
- In general
- Unit tests follow the pattern of
/* Given */ /* When */ /* Then */
- Unit tests follow the pattern of
- JUnit Java tests
- Unit, integration tests for Java code
- Test classes are in a
testpackage- Eg. for
ch.rswk.util.coreinch.rswk.util.core.test
- Eg. for
- Test resources are in
src/main/resources/fixture - Tests are run during build using
maven-surefire-plugin/maven-failsafe-plugin
- QUnit JavaScript tests
- Unit tests for JavaScript code
- Test suite is in
src/main/js/qunit.htmland test modules are imported and run insrc/main/js/modules/tests.js - Test module is named after the module under test, eg.
src/main/js/modules/client-test.jsforclientmodule - Tests are run in WebDriver UI tests, see below
- WebDriver UI tests
- Selenium WebDriver based tests in Java
WebDriverTestclasses are intestpackage and extend AbstractWebDriver- Tests run as part of JUnit suite
- Bootstrap provisioning tests
BootstrapTestclasses are intestpackage and extend AbstractBootstrapTest- Tests are not run as part of CI builds, but used as a manual tool to test bootstrap modules
- Each test case
- Builds bootstrap module for target environment
- Creates and provisions new server instance with module
- Deletes server instance when done
Commit
Checklist:
- Tests green?
- Tests for new code?
- Changed configurations?
- Adapt bootstrap configurations
- Documentation updated?
- Release notes updated?
Commit message format:
Most important thing I did in this commit, a good summary of my changes.
- List of things that changed
- More stuff
Release
platform- Fossil branches
trunk- Development happens here
- Always compiles
- Tests are green
stable- Manually merge/graft commits from
trunk - No development, just minor commits (release notes, fixes)
- Manually merge/graft commits from
- Creating more branches is always an option
- CI jobs
platform_trunk- Build, test and deploy contents of
trunkas SNAPSHOT version to Nexus
- Build, test and deploy contents of
platform_stable- Build, test and deploy contents of
stableas release version to Nexus - Manual steps are required in preparation of and after a release, see Cheatsheet / Release
- Build, test and deploy contents of
- Fossil branches
platform-bootstrap(Already removed as part of thepl3rewrite)- Fossil branches
trunk- Root directory per customer, eg.
customerone - One root directory for the organization itself, eg.
mycompanyfor infrastructure and development bootstrap modules - Root directories contain one directory per bootstrap module, eg.
bootstrap-myapp-server - Bootstrap module directory contains the Jenkins job configuration, build/provision JSON configs and optional customizing files
- Root directory per customer, eg.
- CI jobs
- One per bootstrap module
- Parameters for build (update/install), customer, etc.
- Fossil branches
Versioning
platform versioning is based on Semantic versioning. Major is not expected to
change until the current rewrite. When Minor changes, it's a good idea to check
the release notes. Changes in Patch means internal changes, fixes or new features
that should be backwards compatible.
Consider using a versioning scheme that fits your development style:
Setup How-To
Requirements:
dev-vmdeveloper machine- Ubuntu 22.10 or Windows 11
- 4+ CPU cores
- 4+ GB RAM
- 20 GB disk space
dev-serverdevelopment server- Ubuntu 22.04.01
- 4 CPU cores
- 8 GB RAM
- 100+ GB disk space (depending on number of stored/archived artifacts)
Bootstrap dev-vm
Setup and bootstrap VM
Setup a fresh VM, eg. using VirtualBox, using one of the ISOs:
- Ubuntu 23.10
- Windows 11
- Download Windows 11 Disk Image (ISO)
- Installation
I don't have a product key>Windows 11 Pro- Setup for work
Sign-in options>Domain join
Once the OS installation is finished, use the following scripts to start the bootstrap process.
- Ubuntu terminal
- Download bootstrap module
cd /var/tmp wget -O bootstrap.zip 'https://rswk.ch/bootstrap-dev-vm-nix.zip' - Unzip module
sudo apt install -y unzip unzip bootstrap.zip -d bootstrap cd bootstrap chmod +x bootstrap.sh - Execute module
sudo ./bootstrap.sh
- Download bootstrap module
- Windows PowerShell/Terminal as Admin
- Download bootstrap module
mkdir C:\tmp cd C:\tmp Set-Variable ProgressPreference SilentlyContinue Invoke-WebRequest -Uri 'https://rswk.ch/bootstrap-dev-vm-win.zip' -OutFile bootstrap.zip - Unzip module
Expand-Archive -LiteralPath bootstrap.zip -DestinationPath bootstrap cd bootstrap - Execute module
Set-ExecutionPolicy Unrestricted .\bootstrap.ps1
- Download bootstrap module
- In case of errors, check
bootstrap.login thebootstrapdirectoryERROR restart point created. see log for errors, fix errors, then run bootstrap script again to resume execution - Wait for bootstrap process to finish
executeModule done, elapsed=...
Continue with the manual steps to complete the setup.
Explore platform-archetype
Before we dive into the example application, let's make sure everything is setup
correctly by running the meta-admin web application:
- In IDEA, look for module
meta-admin-testinframework/meta- Open
WebDriverTestclass- In
indexmethod, add breakpoint on line first of line afterrunUiTest - Make sure break point suspend option is set to
Thread
- In
- Click on run symbol for
indexmethod and selectDebug index() - Two browser windows should open, with Chromium opening the
meta-adminclient at https://localhost.platform.rswk.ch:9090/meta-admin/src/main/resources/ - Login with user
platand passwordplat TODO do something in admin?
- Open
- Abort the test case
With that done, we can explore the archetype myapp application. It's part of
platform-archetyp, the reference for a repository based on platform.
- Run a full Maven build of
platform-archetypewithmvn -DskipTests- Linux
~/fossil/platform-archetype - Windows
C:\fossil\platform-archetype
- Linux
- In IDEA, use File > Open and select the directory we used above
- Note that we have to repeat most of the project setup steps in
Project StructureandSettings, like we did for theplatformrepository
- Note that we have to repeat most of the project setup steps in
- Explore
myapp-core,myapp-engine,myapp-testmodules inapplication/myapp - In Visual Studio Code
- Explore
platform-archetype/application/myapp/myapp-web/src/main/resourcesfor the client application
- Explore
- Now everything is ready to try, extend and test the
myappapplication - Debug
WebDriverTest#indexlike we did inmeta-admin-test - Login to the client application using same login as we did for
meta-admin - Open the client in a second tab, then click on one of the tiles and the clients should synchronize
TODO explain how things work together. model, backend service, client, tests, etc.
In addition to platform-archetyp, there is also a reference bootstrap source
repository in platform-archetype-bootstrap containing bootstrap module customizations.
Bootstrap watchdog-server
- Watchdog is a central
meta-engineinstance withmeta-adminclient that is used for monitoring and maintenance for othermeta-enginebased servers in an organization - It is not required for other engines to work, but it is required for automated
provisioning of bootstrap modules, either for testing with
BootstrapTestor for Jenkins jobs that deploy bootstrap modules - Setup Ubuntu VM, eg. with Vultr or any cloud computing provider
- Setup a DNS A record for domain you control that points to the IP of the VM
- Vultr offers a free DNS service
- Example domain used for watchdog server is
watchdog.mycompany.ch- Replace all instances in
platform-archetype-bootstrapof this with a domain you control and would like to use
- Replace all instances in
- Build
bootstrap-watchdog-servermodule in IDEA- In
platformproject underframework/bootstrap/bootstrap-core, right click onMain.java> Modify Run Configuration - Name
Main Watchdog - Program arguments (On Linux, use
~/fossilinstead of `C:/fossil)-build C:/fossil/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/build.json -customize C:/fossil/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server -merge C:/fossil/platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/bootstrap_install.json -module C:/fossil/platform/bootstrap/bootstrap-watchdog-server - Environment variables for variables in
bootstrap.jsonEMAIL_PASSWORD=my_email_password; WATCHDOG_PW=my_watchdog_user_pw;
- In
Upload and bootstrap watchdog-server
- Upload
platform/bootstrap/bootstrap-watchdog-server/target/bootstrap-watchdog-server-VERSION.zipto server's/var/tmpdirectory - Unzip module
sudo apt install -y unzip find /var/tmp -name 'bootstrap-*.zip' -exec unzip {} -d /var/tmp/bootstrap \; cd /var/tmp/bootstrap chmod +x bootstrap.sh - Execute module
sudo ./bootstrap.sh meta-adminshould be available athttps://watchdog.mycompany.ch. Login with userwatchdogand passwordmywatchdogpw- To have a different user or additional users for the other engines to use, create
fixture.jsoninplatform-archetype-bootstrap/mycompany/bootstrap-watchdog-server/config/engineto override the default fixtures inplatform/bootstrap/bootstrap-watchdog-server/src/main/resources/bootstrap/config/engine/fixture.json
Bootstrap myapp-server
- Let's try automated provisioning with
BootstrapTestusing the watchdog server we just setup BootstrapTest(which usesProvisionclass under the hood) uses cloud computing provider Vultr to create and deploy VMs. See Vultr Cheatsheet- In IDEA, open
BootstrapTestinplatform-archetype/bootstrap/bootstrap-test- Remove
@Disabledannotation - This test uses
localhost.platform.rswk.chdomain in configuration files. To access the server after installation, open your localhostsfile and temporarily change the entry forlocalhost.platform.rswk.chto the IP of the deployed VM- Linux
/etc/hosts - Windows
C:/Windows/System32/drivers/etc/hosts
- Linux
- Test can also run with real domains
- Setup DNS A record for a reserved IP with Vultr
- Set
keystoreFileinpropecrtiesmap toAbstractBootstrapTest.CERTBOT_KEYSTORE_FILE - Set
serverDomaininpropertiesmap, eg. totest.mycompany.ch - Pass
certbotflag tonewProvisionTestExecuteConfigmethod - Set
provisiontest_vultr_reservedipto the reserved IP ID
- Set
- Setup DNS A record for a reserved IP with Vultr
- Remove
- Right click on
BootstrapTest.java> Modify Run Configuration- Name
BootstrapTest MyAppServer - Environment variables
provisiontest_vultr_apikey=XXX; provisiontest_vultr_firewall=XXX; provisiontest_vultr_reservedip=; provisiontest_vultr_serverpassword=Hu3m4?mFKk2S9!=; provisiontest_vultr_sshkey=XXX; provisiontest_watchdog_engine=watchdog.mycompany.ch; provisiontest_watchdog_password=my_watchdog_user_pw; provisiontest_watchdog_user=watchdog; - All
XXXabove need to be replaced with actual values from your Vultr settings. See Vultr Cheatsheet
- Name
- Set a breakpoint in
myAppServeronassertWatchdogStatuscall - When breakpoint is reached, the server should be up and running and the client
accessible at
https://localhost.platform.rswk.ch(or the domain you specified)- Check your watchdog server. It should have received the installation status of the new server with a log of the bootstrap process
- Resume the debugging session and the test resources are cleaned up. If everything went as expected, the test should continue to provision the next target environment
Bootstrap dev-server
For professional software development, it's recommended to setup a development server where all the source repositories, automated CI/CD builds and binary artifacts can be managed/stored.
- Setup Ubuntu VM, eg. with Vultr, another cloud computing provider, or even a local hypervisor software if you want to run the server on premise
- Setup a DNS A record for domain you control that points to the IP of the VM
- Example domain used for development server is
dev.mycompany.ch- Replace all instances of
dev.mycompany.chinplatform-archetype-bootstrapwith the domain you control and would like to use
- Replace all instances of
- To just test the setup of a development server, you can run
BootstrapTest#devServerinplatform/bootstrap/bootstrap-test, like we did withmyapp-server - Build
bootstrap-dev-servermodule in IDEA- In
platformproject underframework/bootstrap/bootstrap-core, right click onMain.java> Modify Run Configuration - Name
Main DevServer - Program arguments
-build C:/fossil/platform-archetype-bootstrap/mycompany/bootstrap-dev-server/build.json -customize C:/fossil/platform-archetype-bootstrap/mycompany/bootstrap-dev-server -module C:/fossil/platform/bootstrap/bootstrap-dev-server - Environment variables used in
boostrap.jsonNEXUS_PW=my_nexus_admin_password;
- In
Upload and bootstrap dev-server
- Upload
platform/bootstrap/bootstrap-dev-server/target/bootstrap-dev-server-VERSION.zipto server's/var/tmpdirectory - Unzip module
sudo apt install -y unzip find /var/tmp -name 'bootstrap-*.zip' -exec unzip {} -d /var/tmp/bootstrap \; cd /var/tmp/bootstrap chmod +x bootstrap.sh - Execute module
sudo ./bootstrap.sh
Continue with the manual steps to complete the setup.
Turtles all the way down
- At this point we have setup
- Developer machine
- Development server
- Watchdog server
- But all of this was done from the local developer machine. With the development server, we can now store and build everything from a central place and get other people setup as well
- Clone the
mycompanyandmycompany-boostraprepositories to your local machine- Copy contents of
platform-archetypetomycompanyandplatform-archetype-bootstraptomycompany-boostrap - Rename artifacts, packages, etc. to match your organization
- Commit changes to server
- Copy contents of
- Adapt CI jobs in Jenkins to match your organization
- Repository and job names
bootstrap-dev-vm- Build bootstrap module for developer machines based on the customizing
in
platform-archetype-bootstrap/mycompany/bootstrap-dev-vm
- Build bootstrap module for developer machines based on the customizing
in
bootstrap-dev-server- Build bootstrap module for development server based on the customizing
in
platform-archetype-bootstrap/mycompany/bootstrap-dev-server
- Build bootstrap module for development server based on the customizing
in
bootstrap-watchdog-server- Build bootstrap module for watchdog server with customizing in
platform-archetype-bootstrap/mycompany/bootstrap-watchdog-server- Set
BUILDparam toinstallto build the module. Must be manually installed on a new watchdog server, since provisioning depends on a watchdog server - Set
BUILDparam toupdateto build the module and deploy it to existing watchdog server
- Set
- Build bootstrap module for watchdog server with customizing in
bootstrap-myapp-server- Example to build and provision a customer
myappserver.- Set
CUSTOMERparam tocustomeroneto build job with customizing inplatform-archetype-bootstrap/customerone/bootstrap-myapp-server - Set
BUILDparam toinstallto provision a new server using Vultr - Set
BUILDparam toupdateto update an existing server to a new version
- Set
- Example to build and provision a customer
bootstrap-myapp-cd- Example for continuous delivery job. Updates all
myappservers that run compatible, old version to a new release version
- Example for continuous delivery job. Updates all
Cheatsheet
Keystores
- Java keystore list certificates
keytool -v -list -keystore keystore.pfx - Java keystore import existing keystore (see below how to create PFX from PEM files)
keytool -importkeystore -deststorepass 123 -destkeystore keystore.pfx -srckeystore tmpkeystore.pfx -srcstoretype PKCS12 - Java keystore remove certificate
keytool -delete -alias server -keystore keystore.pfx - Install OpenSSL, eg. see bootstrap definition
platform/bootstrap/bootstrap-common/src/main/resources/bootstrap/definitions/openssl.json - Configuration
set RANDFILE=C:\data\openssl.rnd set OPENSSL_CONF=C:\tool\httpd\conf\openssl.cnf - Convert PEM certificate and key to Java keystore
openssl pkcs12 -export -in server.crt -inkey server.key -out tmpkeystore.pfx -name server -password pass:123 - Export PEM certificates from Java keystore
openssl pkcs12 -in keystore.pfx -out keystore.pem -nodes - Self signed CA and server certificate with SAN
- Adapt configuration in
openssl.cnf[ req ] req_extensions = v3_req [ v3_req ] basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment subjectAltName = @alt_names [ alt_names ] DNS.1 = myapp.com [ v3_ca ] authorityKeyIdentifier = keyid,issuer basicConstraints = critical,CA:true,pathlen:3 keyUsage = critical, cRLSign, keyCertSign nsCertType = sslCA, emailCA subjectKeyIdentifier = hash - Generate certificate
openssl genrsa -out ca.key 2048 openssl req -new -x509 -extensions v3_ca -days 720 -key ca.key -sha256 -out ca.crt openssl genrsa -out server.key 2048 openssl req -extensions v3_req -sha256 -new -key server.key -out server.csr openssl x509 -req -extensions v3_req -days 720 -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -extfile C:\tool\httpd\conf\openssl.cnf
- Adapt configuration in
- Links
Java
- Create runtime from JDK
- Download JDK
- Generate list of all
.jmodfiles injmoddirectory- Eg. with
lsorjava --list-modules
- Eg. with
- Run jlink
- Start shell in
bindirectory - Replace
XXXbelow with comma separated list of all modules (without.jmod) - Linux
./jlink --module-path ../jmods --add-modules "XXX" --output /var/lib/platform/tool/java17 - Windows
.\jlink.exe --module-path ..\ --add-modules "XXX" --output C:\tool\java17
- Start shell in
- Create 7z archive from content of
java17directory- Linux
java17-nix.7z - Windows
java17-win.7z
- Linux
- Upload to
platformdev server
- Zulu Mission Control
- Open
zmc.iniand add path to Java as first argument- Linux
-vm /var/lib/platform/tool/jdk17/bin - Windows
-vm C:/tool/jdk17/bin/server/jvm.dll
- Linux
- Open
Links
- https://anthonyhobday.com/sideprojects/saferules/
- https://app.diagrams.net/
- https://basecamp.com/shapeup
- https://excalidraw.com/
- https://exiftool.org/
- https://freesound.org/
- https://input.fontbureau.com/download/
- https://regex101.com/
- https://regexper.com
- https://www.shellcheck.net/
- https://www.vultr.com/resources/developers/
Logging
- Java
- Tests in
platformframework/util/util-core-test/src/main/resources/logback-test.xml
- Tests in Engine modules
myapp-test/src/main/resources/logback-test.xml
- In general
- Add a
logback.xml(orlogback-test.xml) tosrc/main/resources
- Add a
- Runtime
- For
meta-engineinengine.jsonto overwrite the level of a logger"logging": { "loggers": { "ch.rswk.meta.engine": "DEBUG" } }
- For
- Tests in
- JavaScript based on
util-web- In
config.jssetdebug: true - QUnit tests
Logger.useDefaults({ defaultLevel: Logger.DEBUG }); - Clients using
meta-client/loggingwithmeta-admin- Notifications > Send client notification
- Any channel the client is subscribed to
- Action
INFO - Data
{"logLevel": "DEBUG"}
- Notifications > Send client notification
- In
Lucene
- Inspect index with Luke
- Download latest Lucene distribution
- Extract ZIP and set
JAVA_HOMEin Luke shell scripts- Linux
luke/luke.shset$JAVA_HOME/bin/java - Windows
luke\luke.batset%JAVA_HOME%\bin\java
- Linux
- Run shell script
- Re-index existing index, eg. after new Lucene version
- Use
clonetask fromEngineUpdater- Using
engine-clone.jsonfrom a bootstrap module - Using IDEA JUnit
MainTest.updatetest method with changednewMetaServiceTestConfig(see below)
- Using
- Use
- Re-index existing index, eg. after changes to meta model
- Implement
MetaUpdater/MetaUserUpdaterto migrate existing entities with theclonetask - See
EngineUpdaterSchemaTest.java
- Implement
- Test with existing index
- Get copy of index you would like to run locally
- Manually create a
7z-file from an existingworkdirectory - Or backup and download index with
meta-admin>File>Search backups
- Manually create a
- Upload in local
meta-admin>File- Enter correct tags, eg. for
MetaService:backup,ch.rswk.meta.engine.MetaService
- Enter correct tags, eg. for
- Use restore function on uploaded file
- Get copy of index you would like to run locally
- Alternative way to test with existing index
- Extract/copy index to a local directory, eg.
C:/tmp/index/work - In
MetaServiceTest.newMetaServiceTestConfig, setluceneIndexDirto parent directory, eg.C:/tmp/index - When running
MetaServiceTest, make sure to disablecleanupinteardown
- Extract/copy index to a local directory, eg.
Maven
- Default goals for
platformparent are set toclean install - Only build current module and not full reactor
mvn -N - Run specific test
mvn -DfailIfNoTests=false -Dtest=MyTestmvn -DfailIfNoTests=false -Dtest=ch.rswk.meta.core.*.*Test
- Skip tests
mvn -DskipTests - Only build certain modules
mvn -pl meta-core,meta-client - Only build certain module, including dependencies
mvn -pl meta-engine -am - List module dependency tree
mvn dependency:tree - Upload third party artifact to Nexus
mvn deploy:deploy-file -DgroupId='ch.othercompany' -DartifactId=theframework -Dversion='1.0.0' -Dpackaging=jar -Dfile='theframework.jar' \ -DgeneratePom=true -DrepositoryId=dev-server -Durl=https://dev.mycompany.ch/nexus/repository/thirdparty/- Use
-DpomFile='theframework.pom'instead of-DgeneratePom=trueto provide your own POM
- Use
Release
- Pre release steps
stable- Merge
trunk
fossil merge trunk- Update
src/site/markdown/release.md - Update
src/doc/doc.md - Set release version
npm version "$RELEASE_VERSION" --workspaces mvn versions:set -DnewVersion="$RELEASE_VERSION" -DgenerateBackupPoms=false- Update NPM dependency versions in
package.json
"util-web": "$RELEASE_VERSION"- Tag release
fossil commit -m "Release version $RELEASE_VERSION." --tag "$RELEASE_VERSION"- Merge
- Run
platform_stable - Post release steps
trunk- Merge
stable
fossil merge stable- Update
src/site/markdown/release.md - Update
src/doc/doc.md - Set next version
npm version "$NEXT_VERSION-SNAPSHOT" --workspaces mvn versions:set -DnewVersion="$NEXT_VERSION-SNAPSHOT" -DgenerateBackupPoms=false- Update NPM dependency versions in
package.json
"util-web": "$NEXT_VERSION-SNAPSHOT"- Commit
fossil commit -m "Prepare version $NEXT_VERSION-SNAPSHOT."- Merge
- Run
platform_trunk
Vultr
- Account
- API
- Enable API and store key somewhere safe
- Used in provision config
- Add IP/subnets that are going to call the API to access control (dev machines, dev servers, etc.)
- Enable API and store key somewhere safe
- SSH Keys
- Generate new keys
- Linux
ssh-keygen -f ~/.ssh/vultr_rsacat ~/.ssh/vultr_rsa.pub
- Windows
- Bitvise SSH Client > Client key manager > Generate new
- Enter passphrase > Generate
- Export > Export public key > OpenSSH format
- Linux
- Add SSH Key
- Copy
idquery parameter from URL and store somewhere safe- Used in provision config
- Generate new keys
- API
- Products > Firewall > Add Firewall Gruop
- Accept SSH from your IP
- Accept MS RDP from your IP
- Accept HTTP, HTTPS from anywhere
- Copy
Group IDand store somewhere safe- Used in provision config
- Products > Network > Reserved IPs
- Add for domains with fixed DNS A records and store
- Copy
idquery parameter from URL and store somewhere safe- Used in provision config
Web
- Development
- In
WebDriverTest, put breakpoint on first line afterrunUiTest(...). In IDEA, make sure to set breakpointSuspendtoThread - Use Chromium/Firefox instance opened by the
WebDriverTestfor client development - If you need to access any service
runtime.getServices(MockGatewayService.class).get(0)
- In
- Service workers
- Force reload and refresh without using dev tools
- In
config.js/config.jsonchange version - In
worker-init.jschange version in comment on first line
- In
- Force reload and refresh without using dev tools
- URL for tests with
ch.rswk.util.web.test.WebServerhttps://localhost.platform.rswk.ch:8088/qunit.html - URL for tests with
meta-engineandHttpServicehttps://localhost.platform.rswk.ch:9090/ - Execute action via REST-like API from shell
- Linux Bash
#!/bin/bash EXECUTE_REQUEST='{"actionId":"myaction","param":{},"reqt":"ecq"}' RESPONSE=$(curl -s --fail --header "Accept:application/json" \ --header "Authorization:Bearer $(curl https://myapp.mycompany.ch/rs/token -s --data "user=myuser" --data "password=pw123")" \ --header "Content-Type:application/json" --data "$EXECUTE_REQUEST" \ --request POST https://myapp.mycompany.ch/rs/execute) if [ $? -eq 22 ]; then echo "Error" else echo "$RESPONSE" fi - Windows PowerShell/Terminal
$executeRequest = @{ actionId = 'myaction' param = @{} reqt = 'ecq' } $headers = @{ 'Accept' = 'application/json' 'Authorization' = $("Bearer " + (Invoke-WebRequest -Body @{user='myuser';password='pw123'} -Method 'POST' -Uri https://myapp.mycompany.ch/rs/token)) 'Content-Type' = 'application/json' } try { $response = Invoke-WebRequest -Body ($executeRequest | ConvertTo-Json) -Headers $headers -Method 'POST' -Uri https://myapp.mycompany.ch/rs/execute echo $response } catch { $err = $_.Exception echo $err.Response }
- Linux Bash
Software references
Disclaimer
- Highcharts is only free for non-commercial use. If you plan to use it in a commercial setting, please buy a license
- SMS delivery in GatewayService depends on an external provider based in Switzerland: iNetWorx SMS-Gateway. Please contact them to order a SMS package. No affiliation with iNetWorx, just happy customers for many years
- Automated provisioning of bootstrap modules depends on having an account with Vultr. We would like to add more providers in the future, but are happy right now with the simplicity, functionality and pricing provided. Again, no affiliation with Vultr
