dizmoGen

What is dizmoGen?

dizmoGen helps you to write, build, test and deploy dizmos. It extends Yeoman and gulp to speed up and automate your dizmo development workflow. dizmoGen is an open-source project, hosted at Github.

Prerequisites

How does dizmoGen work?

DizmoGen is a generator that creates project folders. A generated folder will contain the skeleton for a new dizmo. It also contains the build scripts needed to turn the source code into a packed and installable dizmo artifact (*.dzm).

The generator itself is based on the Yeoman generator toolkit (which in turn is based on Node.js). Neither dizmoGen nor Yeoman is required to build your dizmo after the initial generation of a project.

Use any text editor or IDE to edit the generated source code. Once ready, you can use the build scripts to turn your project into a correctly formatted and packed dizmo.

Invoking npm run <script> uses internally a Node.js based tool named gulp, which orchestrates the entire build process to turn your source code into a dizmo. Possible <script> commands are build, lint, clean, etc.

As the totality of all build scripts with their dependencies account for more than 100 MBytes, the dependencies are excluded when pushing a source folder to a remote Git repository. Therefore, it is necessary to install a dizmo project that you get from an external origin. This is done within the build process automatically. It can also be triggered manually by invoking npm install from within the project folder, which will create a folder named node_modules. The node_modules folder should never be included in the Git repository.

The options and commands available to build, test and deploy dizmos are upgraded from time to time. Therefore, it may become necessary to update the relevant build scripts using dizmoGen (and Yeoman) again. Check the «Upgrading the Build System» section below on how to actualize your version of dizmoGen and the build scripts in your dizmo project.

Installation

First, install Yeoman and generator-dizmo using npm, assuming that you have Node.js pre-installed:

(sudo) npm install -g yo
(sudo) npm install -g @dizmo/generator-dizmo

On most operating systems the -g option (shortcut for --global) requires superuser (administrator) rights. Therefore, on Unix-like systems you might be required to run the above commands using sudo, i.e. sudo npm install -g yo and sudo npm install -g generator-dizmo. Alternatively, you can run them directly from your operating system's superuser account.

Quick start

Invoke the dizmo generator with a name of your choice, for example, my-dizmo and answer a few questions:

yo @dizmo/dizmo my-dizmo

After a successful build, drag and drop the ./build/MyDizmo-0.0.0.dzm file onto dizmoViewer. You should see the front side of the dizmo with Hello World! written on it. The name parameter is optional and can be changed at the prompt.

Furthermore, calling yo @dizmo/dizmo is equivalent to invoking the default generator with yo @dizmo/dizmo:app.

To list all possible arguments and options of the generator, enter:

 yo @dizmo/dizmo --help

Caching

Npm uses a built-in cache mechanism to accelerate package installation. There are various configuration options to control the behavior of the cache. Here, we are interested in cache-min:

The minimum time (in seconds) to keep items in the registry cache before re-checking against the registry. [default: 10]

The provided default of 10 seconds is too short, to efficiently make use of caching. Therefore, we recommend setting it to for example a week by running the configuration command:

npm config set cache-min 604800

By setting cache-min to 604800, you ensure that no package with a timestamp younger than a week is checked against the central registry for a possible update. This significantly improves your npm experience.

Furthermore, we suggest to initially run npm cache clean to clear the cache. However, this is not necessarily required as it will simply wipe out your cached packages and ensure that no corrupted cache exists. However, not running the command means that your very first dizmo skeleton generation and corresponding installation of npm packages will take longer than later invocations. By running npm cache ls you can determine which npm packages have already been cached.

Note: It is also recommended to run npm cache clean and clean the cache before an update of the generator-dizmo generator itself.

Questions

When you have invoked dizmoGen, you will be asked a few questions. The terminal should look similar to:

~/my-dizmos $ yo @dizmo/dizmo

     _-----_
    |       |    .--------------------------.
    |--(o)--|    |  Welcome to the awesome  |
   `---------´   |     dizmo generator!     |
    ( _´U`_ )    '--------------------------'
    /___A___\
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

? Name your dizmo: MyDizmo
? Describe it: My Dizmo
? And its bundle ID? com.example.my_dizmo
? What's your name? Name Surname
? And your email? name.surname@mail.net

The dizmo generator asks you some questions -- let's have a look at them:

? Name your dizmo: MyDizmo

If no dizmoName argument is provided, then by default MyDizmo will be suggested. This name will be used to create a project folder in the current directory. For example, if the name of your dizmo is MyDizmo, then the folder name will be my-dizmo/.

? Describe it: My Dizmo

You should provide a short succinct description of your project. A suggestion is provided by default that is based on the current name of the dizmo.

? And its bundle ID? com.example.my_dizmo

Each dizmo is required to have a unique bundle.identifier. This is the name of the bundle that each dizmo instance will belong to. For example, each sticky-note dizmo would have the same com.dizmo.stickynote bundle ID, but have different dizmo IDs.

Choose the domain of your company as a prefix. Use reverse notation, with the top-level domain name (e.g. com or org) preceding the rest. Then append a name related to the dizmo.

? What's your name? Full Name

Provide your full name, to designate yourself as the author of the project. By default, the current GIT user name – if available – or OS login will be used directly without actually prompting for the name. If you enter another name here, that name will be remembered and used as the default on your next invocation of yo @dizmo/dizmo.

The entry will be stored once the project skeleton is set up in package.json under person.name. For multiple contributors, see the npm:package.json documentation, section people-fields-author-contributors.

? And your email? my@email.net

Provide your email so that people can reach out to you for feedback, bug reports, etc. If available, the generator uses the GIT user email by default. Otherwise, enter your e-mail using the MAIL environment variable.

The entry will be stored in package.json under person.email. For multiple contributors, see again people-fields-author-contributors.

Upgrading the Build System

The build system of each dizmo is saved directly within a project. Therefore, we need a build system upgrade mechanism so that projects can exist.

First, we have to upgrade generator-dizmo by running:

npm upgrade -g @dizmo/generator-dizmo

Then within an existing project's main folder, we can execute:

yo @dizmo/dizmo --upgrade

It's also possible to only invoke yo @dizmo/dizmo, in which case every conflict between the existing and new files and folders need to be manually signed-off by the user. However, manual upgrading can be onerous since all conflicts need to be decided on. With the upgrade mechanism, you only need to decide on the conflicts with respect to the build system.

Note that with the yo @dizmo/dizmo --upgrade command, only the build system of the actual project is upgraded. Non-build related files and folders remain untouched.

Skeleton

After you have answered the last question, the generator will create the project's skeleton. If you have the tree command installed on your operating system, then you can visualize the directory structure:

my-dizmo $ tree
.
├── .eslintrc.json
├── assets
│   ├── Icon-dark.svg
│   ├── Icon.svg
│   ├── locales
│   │   ├── translation.de.json
│   │   └── translation.en.json
│   └── Preview.png
├── gulp
│   ├── package.js
│   └── tasks
│       └── *
├── gulpfile.js
├── help
│   └── en
│       ├── help.md
│       └── placeholder-400x275.png
├── LICENSE
├── package.json
├── README.md
└── src
    ├── index.html
    ├── index.js
    ├── lib
    │   └── i18n-*.min.js
    └── style
        └── style.scss

Let's have a look at each of the top-level files and directories:

Package manager: package.json

Dizmos use npm as a package manager; to thoroughly understand its functionality, please consult "What is npm?" and work through the 15 small video-based tutorials in the Getting Started section.

Dizmo section

In addition to the default entries of npm the package.json file contains a dizmo section:

"dizmo": {
    "settings": {
        "attributes": {
            "settings/usercontrols/allowresize": true
        },
        "bundle-identifier": "com.example.my_project",
        "bundle-name": "My Project",
        "category": "",
        "height": 360,
        "tags": [
            "my-project"
        ],
        "width": 480
    },
    "store": {
        "host": "https://store-api.dizmo.com"
    },
    "build": {
        "lint": true,
        "minify": {
            "markups": {
                "htmlmin": true
            },
            "scripts": {
                "sourcemaps": false,
                "obfuscate": true,
                "uglify": true
            },
            "styles": {
                "sourcemaps": true,
                "sass": true
            }
        }
    }
}

And here is a list of available options:

It is not recommended to obfuscate vendor scripts and polyfills since the obfuscated code can be 15-80% slower and the files are significantly larger. Hence, (large) third party libraries should be referenced with a <script href="lib/jquery.min.js"> tag directly from within your HTML markup. This pertains to libraries such as jquery.min.js that are embedded into a dizmo under, for example, the src/lib/ directory. Thus, you will avoid the libraries being obfuscated and there will not be any performance penalty.

However, any library that is installed into a dizmo project, e.g. via npm install --save jquery, and then referenced as var $ = require('jquery') from your source code will be collected by the build system and subsequently also obfuscated. This might be desirable, for example, if the third party library is company internal code and you wish to use it in your dizmo. This arrangement will then protect your company's intellectual property while offering you the desired functionality.

Default Configuration

The dizmo section in package.json can be extended with default values. The default values have to reside in .generator-dizmo/config.json, which itself has to exist in any of the parent directories to package.json:

{
    "dizmo": {
        "deploy-path": "..", "store": {
            "host": "https://store-api.dizmo.com",
            "user": "..",
            "pass": ".."
        }
    }
}

The configuration is hierarchical and recursive, meaning that a .generator-dizmo/config.json file can be saved in any parent directory of the current project, all of which are then merged during the build dynamically into package.json. There, configuration values from files in the lower levels (meaning closer to package.json) have precedence.

Yeoman: Managing Configuration

As an alternative to .generator-dizmo/config.json, the .yo-rc.json file can be used to store default configuration values. See managing configuration for further information.

NPM scripts

Before running any script, please ensure that npm install has been executed and that the dependencies beneath node_modules are up to date.

Please read first npm#scripts. In each package.json the following scripts are available:

CLI options

The build process supports command-line arguments to quickly override some of the configurations in package.json. It's important to realize that this CLI support is directly integrated via the underlying primitives, i.e. linting can be enabled or disabled via --lint or via --no-lint. This argument can be provided to any script which depends on the linting step.

In many cases, the arguments are boolean flags which can enable or disable a certain build step (like linting or minification). But many can also accept specific configuration objects -- like JSON. For example, the linting of JavaScript is controlled via an eslint specific configuration object.

However, for daily usage, the default settings should be more than enough. Simply using the CLI arguments as boolean flags to enable or disable a particular step will do the job. Furthermore, please note, for example, that the corresponding step will still be shown during the build when linting is disabled, but it won't perform any actual linting.

Linting: --lint or --no-lint

On the command-line, linting can be enabled by providing --lint, and it can be disabled by providing --no-lint. These flags will override (or merged with) the configuration from package.json:

{
    "dizmo": {
        "build": {
            "lint": true
        }
    }
}

Above it is specified that the linting step should be executed by default for the given project. Hence, the following will lint and build the dizmo:

npm run build

To stop the build engine from linting, either set the lint entry in package.json to false or override it directly on the CLI:

npm run build -- --no-lint

The double hyphen after npm run build is necessary. It tells NPM to forward the --no-lint argument to each of the build steps, that together will build (i.e. build) the dizmo. Alternatively, you can provide the script name after the initial double hyphen if you don't like the four consecutive hyphens:

npm run -- build --no-lint

Or more verbosely below you see in its clearest form, how the build script is run with the additional argument --no-lint:

npm run-script -- build --no-lint

Conversely, if you explicitly want to enforce linting then you can execute:

npm run -- build --lint

As mentioned, this is in general not required since package.json should by default have linting enabled. However, if you are not sure that linting is enabled and you want to enforce it, then it makes sense to provide the --lint flag explicitly. This could be the case if, for example, you are putting together a build environment.

The specific configuration objects for controlling eslint, coffeelint and tslint can be looked up via their respective documentation. Some very simple examples have been provided below to demonstrate the corresponding capability to override default settings (and/or any existing configuration object in package.json).

npm run build -- --lint='{"rules":{"no-unused-vars":0}}'
npm run build -- --lint='{"rules":{"no-unused-vars":1}}'
npm run build -- --lint='{"rules":{"no-unused-vars":1}}'

The build process will not fail in case of an error in the configuration objects, effectively making it equivalent to a warning. If such behavior is not desired, then the lint.js Gulp task should be modified to stop the build process upon a linting error.

Minification: --minify or --no-minify

Providing the --minify option on the CLI will ensure that the scripts, styles, and markup are automatically minified and obfuscated, where obfuscation operates only on the scripts:

npm run build -- --minify

Please note that source maps are not created by default. This is to avoid accidental leaks of potential intellectual property. However, source maps can be enabled by appending the --sourcemaps flag:

npm run build -- --minify --sourcemaps

It is also possible to suppress a minification, for example in the case that it is enabled via package.json:

npm run build -- --no-minify

Furthermore, it is possible to control the five sub-steps of minification independently of the general --minify flag. Use then the five sub-steps of minification: (a) markup minification, (b) styles minification, (c1) scripts obfuscation, plus (c2) minification, and also (d) source maps generation -- where the latter, however, needs to be explicitly enabled.

npm run build -- --htmlmin --sass --obfuscate --uglify --no-sourcemap

The set of arguments above is equivalent to the --minify flag, given default package.json build settings.

Furthermore, each argument can be negated:

npm run build -- --no-htmlmin --no-sass --no-obfuscate --no-uglify

Each flag can also accept an optional configuration object to control the corresponding minification, obfuscation and/or source map generation step in detail:

npm run build -- --minify --htmlmin='{"collapseWhitespace":true}'
npm run build -- --minify --sass='{"outputStyle":"compressed"}'
npm run build -- --minify --obfuscate='{"compact":true}'
npm run build -- --minify --uglify='{"mangle":true}'
npm run build -- --minify --sourcemaps='{"loadMaps":true}

In general, using --minify (or --no-minify) combined with the --sourcemaps (or --no-sourcemaps) CLI options should be enough. Only if explicit control is required, using the --htmlmin, --sass, --obfuscate or --uglify flags might be necessary. However, use the flags carefully. These flags should only be provided with configuration objects if you are not happy with the defaults.

Upload

Dizmo offers a dizmoStore where dizmos can be uploaded to. Besides package.json (or .generator-dizmo/config.json) or environment variables, upload arguments like the host and user name plus password can also be provided via the CLI:

npm run upload -- --host=https://store-api.dizmo.com --user='..' --pass='..'

By default, npm run upload tries to upload and publish an uploaded dizmo.

However, it is possible to skip the publication step by running:

npm run upload -- --no-publish

And then to publish it in a subsequent step:

npm run upload -- --publish

However, the command above assumes that the actual upload has already been performed. Executing the command without having uploaded a dizmo previously will fail since there would be no uploaded dizmo to publish.

Build

Once your dizmo is built, a build/ folder with the following structure is created:

my-dizmo $ tree build/
build/
├── MyDizmo
│   ├── Icon-dark.svg
│   ├── Icon.svg
│   ├── Info.plist
│   ├── Preview.png
│   ├── assets
│   │   ├── Icon-dark.svg
│   │   ├── Icon.svg
│   │   └── Preview.png
│   ├── help.zip
│   ├── index.html
│   ├── index.js
│   ├── lib
│   │   └── i18n-*.min.js
│   └── style
│       └── style.css
└── MyDizmo-0.0.0.dzm

In dizmoViewer only the dizmo bundle with the highest version number is cached. Therefore, it is important to increase the version when releasing a dizmo to your audience. However, simply changing the version suffix in the *.dzm file name will not work. The version is required to be set in MyDizmo/Info.plist. This happens automatically, based on the version information in package.json.

Dizmo instantiation

Drag and drop the MyDizmo-0.0.0.dzm artifact onto dizmoViewer to instantiate a corresponding dizmo. If it has not been already installed, it will be installed to the default location as well.

Extended sub-generators

Once you are more familiar with dizmo development, you can go and try out the dizmo:sub-coffeescript and dizmo:sub-typescript sub-generators.

Miscellaneous

GIT initialization

Invoke a generator combined with the --git option:

yo @dizmo/dizmo my-dizmo --git

The project folder that is created will now be named my-dizmo.git, and it will be initialized as a GIT repository. However, no commits will be performed. Furthermore, this will only work if the git command is accessible.

Dependency management

All (sub-)generators support dependency management using Node modules. You can structure your dizmo code using require, exports and module.exports objects.

Furthermore, you can install external third party libraries and reference them directly with require. For example, to use jQuery, run: npm install --save jquery. Then in your code you can get a reference with: var $ = require('jquery');. If you want to remove an installed library, run: npm remove --save jquery.

This approach works well as long as the external libraries are not too large. If you are using a large external library, the build process may take longer. In such cases, you should use the incremental builder using the watcher: npm run watch. Or, you can simply drop a library into the src/lib/ sub-directory and reference it accordingly via a corresponding <script> tag in the index.html markup.

Troubleshooting/FAQ

The generator-dizmo package seems to be deprecated

Yes! It has been migrated to @dizmo/generator-dizmo, which means to continue to receive updates for the generator. A re-installation is required:

(sudo) npm uninstall -g generator-dizmo
(sudo) npm install -g @dizmo/generator-dizmo

Furthermore, any yo dizmo invocation needs to be changed to yo @dizmo/dizmo, to upgrade older projects to run the command with the @dizmo scope below:

yo @dizmo/dizmo --upgrade

Did I forget to run npm install?

If npm install is not run before attempting to build a dizmo, then a message similar to the one below might be produced:

error argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "build"
error code ELIFECYCLE
error MyDizmo@0.0.0 build: `node ./node_modules/gulp/bin/gulp.js`
error Exit status 1
error Failed at the MyDizmo@0.0.0 build script 'node ./node_modules/gulp/bin/gulp.js'.

In such a case, run npm install to ensure that all the required dependencies are installed locally.

Can I run yo @dizmo/dizmo as root?

The Yeoman toolkit strongly discourages the usage of any generator based on it to be run as root. Hence, you will receive the following error:

$ sudo yo @dizmo/dizmo --help
/usr/lib/node_modules/yo/node_modules/configstore/index.js:53
                throw err;
                ^

Error: EACCES: permission denied, open '/root/.config/configstore/insight-yo.json'.
You don't have access to this file.

The same error is thrown, when you run sudo yo -h as well. Also, the behavior is independent of the usage of sudo or directly being logged in as root.

The recommended approach is to create a non-root user account and then run the generator under that account. If you are manually generating a dizmo project, this should be the case anyway, since you would most probably be logged in under a personal user name. However, this might not be the case when running in the context of, for example, an automated build environment. Hence, the need for a special purpose user account.

Does dizmoGen support ES6?

Browsers and the libraries which they are built upon usually lag behind the latest standard. Hence, they fail to provide up-to-date language support. However, the Babel transpiler can take a script written in a modern standard and translate it into backward-compatible JavaScript. DizmoGen includes Babel and thus supports ES6.

Note: If you have older projects and update dizmoGen, ensure that you add ES6 support to .eslintrc.json as following:

{
  "env": {
    "browser": true,
    "node": true,
    "es6": true
  }
}

How to create a .generator-dizmo folder on Windows?

The graphical user interface of Windows does not allow to create a folder named .generator-dizmo. However, it is possible to create one via the command-line interface. For example, using the Windows PowerShell one can run:

PS C:\Users\user> mkdir .generator-dizmo

How to ignore .DS_Store files on Mac OS X?

.DS_Store files are device-dependent and should, therefore, be ignored globally on the developer's device, instead of on a per-project basis. See How to Remove .DS_Store File from a Git Repo on Mac OS X for an excellent discussion of the issue. Also at GitHub Help, see ignoring files to learn about setting up a global ignore list.

© 2018 dizmo AG, Switzerland