ZYME OS is an umbrella term for a collection of systems, rooted in the made up word “Zyme”, an abbreviation of enzyme. It is a principle that can be applied to any type of media production, though the systems built have mainly been designed to produce 1024x1024 pixel images.
The key to the functionality of ZYME OS is recursion, mixed with opinionated interpretation. These two functions create a balanced and continuous growth of novel media output.
The recursive function is simple — in the case of the current ZYME OS architecture, image outputs get fed back into the generation cycle as new inputs. The ratio of inputs to outputs is 4:1. Each of the four inputs has its own influence on the generated output; style, composition, negative embedding, and a “canvas” base to generate on top of.
The principle of opinionated interpretation is how the system maintains its stability. Since the models are forced to “interpret” these input images in latent space and apply them as weighted preferences to the act of creating a “new” image, the cycle does not atrophy. Each output is a new interpretation of the combined inputs, not an addition or subtraction.
By contrast, algebraic manipulation of inputs like addition, multiplication etc. leads to a cascade of influence. For example, if you always slightly multiplied the red values of one image set into another image set, the output image would cascade towards a 100% red canvas in a short number of cycles.
This is the value of the latent interpretation in the system. It is resilient to this type of cascade. The system encourages a high amount of output diversity, while giving control to the user via curation of the input fields.
The Session Configurator
This is the control center of the current system. The session configurator sets the conditions for a “run”, where the starting pool of images for each of the 4 input folders is selected, and a variety of modules are toggled on or off. The module system is opt-in by design, which allows for infinite extensibility.
A new module can be introduced as a modifier to the system at any position in the generation cycle to add a layer of variance. Some of these are scripted events that modify the JSON of the comfyUI patch, others are text modifiers or post-generation visual modifications, like adding a color LUT or a metadata augmentation.
When no modifiers are toggled on, the system runs its primary function of its 4 input, 1 output generation cycle.
Some modules have become standard practice, like recursion, which may be baked into the default mode of operation in the future.
The target for the next version of the session configurator is set to include an “epoch” mode, where 1 month intervals of sessions will be run automatically, and a set of determining factors based on the outputs from the previous epoch, combined with curatorial decisions based on the 12 months of the year will make choices on which modules are assigned to each Epoch.
Curation Methods
A key component of the system is the human-in-the-loop process of curation of outputs. This is where taste enters the picture, and choices are made as to which output images ultimately “graduate” into future sessions. This is how the user can drive the macro forces that determine the stylistic output of the system.
The curatorial anchor is the Pool and Negative_Pool folders, which are two large unstructured folders of images that the session configurator pulls from at random to use as the starting set of inputs. The Pool and Negative_Pool are “pruned” by a human user, adding and deleting images based on taste preference.
This provides a macro level of guidance that puts the user in control of the ambient direction of the images look and feel, without asking too much of the user to be micromanaging the system and its output. The aesthetic direction of the outputs can be as refined or unrefined as the user prefers.
If output images coincidentally have too many human figures in them, or too much of a single color, they can be deleted from the pool of input options, and curated images from previous runs can graduate into being pool candidates for the following sessions.
Future implementations of ZYME OS are set to include curation/selection tools that provide the option for crowd sourced curatorial selection. An example of this would be a front end site with the output folder and a voting system that was publicly accessible via invitation from the session owner.
Local First
ZYME OS runs on local hardware with open source models. ZYME OS does not require a subscription to a service of any kind. It is designed to be a studio tool that works how and when you want it to, at the rate and scale you want it to.
Those who contribute to the curation or architecture of ZYME OS have equal usage rights associated with the outputs. The outputs are seen as source material for other projects. The scale and quantity of the outputs from ZYME OS encourages generosity of access.
Conclusion
The work on ZYME OS will eventually be open sourced as a public repo. It has been the backbone of many projects from our studio, and will continue to provide the image layer for many projects to come.
To see a real-time feed of the most recent output from the ZYME OS server, please go to http://178.156.249.95/.