Specify the result
Describe the required output and its correctness check. Include errors and timeouts in the result record.
Computing efficiency for your workload.
Valen Systems develops Rubian and Kestowv for programmable work, system execution and resource coordination. Evaluate how a workload uses the machine, which work finishes correctly and how a change affects response time and resource demand.
Find what limits useful output.
A workload can spend time executing, waiting for data or competing for shared resources. Review completed work alongside CPU activity, memory use, queues and response time to locate the constraint that matters to the application.
For a batch job, define the completed result and the time available to produce it. For a service, define acceptable responses, concurrent demand and the response-time boundary. These requirements give a runtime change, concurrency adjustment or capacity purchase a concrete test.
Use the operating foundation to inspect and repeat the work.
Rubian provides commands, system observations, projects and job controls. Its project kit gives source, findings, runnable snippets and session notes a consistent place. Operators can retain the inputs and decisions needed to resume an evaluation.
Kestowv’s supplied pilot interfaces run configured workloads and retain result information. A single run checks bounded correctness, readiness or failure handling. An A/B workload compares a baseline and candidate. A soak workload repeats work over a stated duration so the record includes behavior across that period.
Define a comparison your team can repeat.
Choose the operating question and acceptance criteria before running the comparison. Keep the definition of useful work consistent across the baseline and candidate.
Record the conditions
Identify hardware, runtime and application versions, inputs, resource limits and competing activity. State any difference between the compared configurations.
Repeat the comparison
Define warmup, iteration count and trial order. Retain variation and failed runs alongside successful measurements.
Check the operating consequence
Read throughput, response time and resource use together. Include cleanup and recovery behavior where they affect whether the change is acceptable.
Read the result with its tradeoffs.
A candidate may finish more work while increasing response times for some requests. The acceptance criteria determine whether that tradeoff serves the application. Report energy when the environment provides suitable measurements, and identify the measurement boundary.
Our published energy and responsiveness studies keep those measures separate. The energy report presents throughput and energy-per-correct-work alongside response-time and package-power tradeoffs. Those records help explain the method and the questions to examine on another workload.
Match the evaluation to the supplied component.
Rubian’s working environment and Kestowv’s pilot have documented versions, configuration requirements and available interfaces. We establish the component, host environment and instrumentation for the work being evaluated. A result belongs to that configuration; an integrated Project Valux evaluation requires its own system and test.
Troy Mallory’s builtin-tracking contribution provides a separate example of the systems work: JRuby maintainers merged and extended it across runtime fast paths, and it shipped in the JRuby 10.1 release line. The deployment record identifies the contribution and release history.
Start with the decision the evaluation must support.
Bring the workload description, target environment, current measurements and configurations you are considering. Tell us about constraints on data, runtime, access and instrumentation. The question may concern a growing queue, a runtime choice, memory pressure or a proposed hardware purchase.
The Compute Evaluation Plan covers one workload and up to two proposed configurations or approaches. It defines inputs, correctness checks, repeatability requirements and a measurement matrix, with written findings and a review call. Running the benchmark is agreed separately. The offer page contains the current price, delivery terms and complete scope.
An executed evaluation should leave your team with the completed-work results, test conditions, tradeoffs and remaining questions needed to choose the next change.
Bring the workload and the decision.
Tell us what your application must complete and which constraint you need to resolve.