HomeCoinsAlgorand (ALGO)Automating with the Mule framework - Algorand

Automating with the Mule framework – Algorand

- Advertisement -


Brice Rising

Mule is a generalized automation framework for organizing the execution of outlined automation in a declarative yaml format. For some context into why a software like that is helpful, we are going to discover the issue it’s making an attempt to unravel.

The issue

In the present day there are a whole lot, and even 1000’s, of instruments in the marketplace which might be excellent at executing very particular actions which might be helpful as a part of an software’s CICD course of. The issue is that with the intention to use these instruments, engineers want to jot down scripts that execute the options they want, whereas taking very particular care that they execute them within the appropriate order.

Typically, this results in brittle scripts which might be tightly coupled to the applying the developer is engaged on. At Algorand, we work on many alternative stacks to make sure that builders can devour the entire nice options we offer on our platform. Subsequently, tightly coupled scripts won’t work for us at scale.

To resolve this downside, we would have liked a software that may meet the next situations:

  • Automation is decoupled from mission repositories
  • Automation might be consumed in a handy method
  • Automation is persistently executed by each customers and our CICD course of

There are a couple of pipeline applied sciences that nearly meet all three of those situations. The difficulty we have now with utilizing these choices is that we don’t want to be married to anybody pipeline know-how, and likewise there are various issues we want to automate that don’t have to be a part of an software pipeline. Subsequently, we determined these choices weren’t fairly handy sufficient for us.

There are a couple of instruments in the marketplace that meet two of the three situations, however most of those require both entrypoint scripts within the software repositories, which fails the primary situation, or studying a brand new syntax that isn’t easy or light-weight sufficient, which fails the third situation. Some combos of those instruments might handle to satisfy all three situations, however we felt that their characteristic units can be too various for the straightforward use case we have been in search of.

After contemplating these choices we determined to discover constructing our personal resolution to this downside.

Our resolution

Our resolution is known as Mule. Mule is a framework that generalizes the idea of a unit of automation, and lets you manage their execution via an intuitive, declarative yaml format. Under is an evidence of how this abstraction offered by Mule meets the said situations:

  • Automation is decoupled from mission repositories

It permits customers to outline automated duties as python scripts in a single centralized location with none want for scripts within the software repositories.

  • Automation might be consumed in a handy method

It supplies a easy, intuitive yaml language for declaring configurations for automated duties and their execution order.

  • Automation is persistently executed by each customers and our CICD course of

It supplies a cli that may devour any legitimate mule yaml file and execute the automated duties the identical method all over the place

This abstraction has helped us to begin organizing our CICD course of and given us confidence that we are able to carry out the time consuming effort of automating a process precisely as soon as in order that we are able to focus extra time on constructing the Algorand community.

Earlier than moving into tips on how to use mule to execute automation, the next are the important thing ideas of the Mule framework.

Duties

A process is a person unit of automation. You can also make a process for any exercise you want to automate. For a listing of at the moment out there duties, see our process documentation. Particulars on tips on how to write and use customized duties can be out there within the coming weeks.

Jobs

A job is a collection of executable duties, and a process itself for that matter. Whenever you present a job with a listing of duties, the job handles evaluating the chain of dependencies of the duties you want to execute and ensures that each one duties required by the job and their dependencies are executed as soon as within the order that they’re wanted.

On this part we are going to present a collection of examples that can assist you get used to the syntax utilized in a mule.yaml.

If you need to play together with this information, have python 3.7+ put in and run the next command to put in the mule cli.

pip set up mulecli

Howdy mule.yaml

duties:
- process: Echo
message: Message for the console

Right here we have now a mule config file that lists one process and one job.

The duties area is the place you doc process configurations that you will want to execute your jobs. It takes a listing of maps (dicts in python) which might be used to initialize your duties. Each process configuration should have a process area set. That is utilized by mule to seek out the duty you want to be initialize. Each different area is a configuration on your process. On this instance we have now one process referred to as Echo, which merely prints out “Message for the console”.

Subsequent you’ve gotten your jobs area. That is the place you outline jobs and supply them with a listing of duties you want to execute when the job is invoked. The listing within the job’s duties area incorporates the ids of the predefined duties, which on this case is simply the duty definition.

When job-a is invoked, we see the next response:

$ mule -f path/to/mule.yaml job-a
Message for the console

And there we have now our first mule job! Now we are going to take a look at a extra fascinating instance.

Job Dependencies

duties:
- process: Echo
identify: A
message: Message for the console
- process: Echo
identify: B
message: One other message for the console
dependencies: Echo.A

Now we have now two duties that we’re defining. The primary is an Echo process that prints out “Message for the console” and the second is an Echo process that prints out “One other message for the console”. Since we’re utilizing two duties who use the identical process definition, we should present a reputation area to every that differentiates them. When a reputation area is launched to a process, its id turns into task_definition.identify.

Yet one more fascinating factor right here is the dependencies area in Echo.B. This area is used to declare {that a} process is determined by one other process. This manner, when the duty is executed inside a job, some other process it is determined by can be executed as nicely. This area can both be an area delimited string, or a yaml listing of strings. Be mindful of the order you prepare these dependencies, for the reason that dependencies can be executed within the order you introduce them in right here.

When job-b is invoked, we see the next response:

$ mule -f path/to/mule.yaml job-b
Message for the console
One other message for the console

Yet one more factor to notice is that this file is logically equal to the next mule.yaml.

duties:
- process: Echo
identify: A
message: Message for the console
- process: Echo
identify: B
message: One other message for the console
dependencies: Echo.A

Regardless of the association of duties, Mule will consider the chain of dependent duties that have to be executed such that duties get executed as soon as within the order they’re wanted.

Subsequent we are going to look into how one can parameterize your duties

Job configs

Job configs are a option to set fields in your mule.yaml that can overwrite fields set in your process configurations. This manner, you may outline duties that may work otherwise in numerous jobs.

duties:
- process: Echo
identify: A
message: Message for the console

Right here we have now the identical Echo process from the primary instance, however in job-a we have now added a configs area. This configs area takes a map, and this can be utilized to overwrite fields of the identical identify within the duties you’ve gotten outlined. As a result of right here we’re overwriting the message area in all of our duties, we are going to see one thing completely different after we execute job-a.

$ mule -f path/to/mule.yaml job-a
One other message for the console

Now since our Echo process’s message area has been overwritten, we’re seeing mule print out “One other message for the console”. We could supply extra refined methods to overwrite process configs sooner or later, however for now it’s necessary to watch out when deciding on fields that you simply want to overwrite in order that you don’t see unintended penalties in your jobs.

One other factor you are able to do in a jobs config is ready values utilizing your system’s setting variables.

duties:
- process: Echo
identify: A
message: A

Now our Echo process will print out the contents of the ECHO_A setting variable.

$ export ECHO_A='Message for the console'
$ mule -f path/to/mule.yaml job-a
Message for the console

Now that we are able to use jobs to parameterize duties, let’s take a look at how we are able to use different duties to execute our duties!

Job Outputs

duties:
- process: Echo
identify: A
message: One other message for the console
- process: Echo
identify: B
message: '{{ Echo.A.outputs.message }}'

Each time a process is executed, it produces a dictionary of outputs and shops them in reminiscence for duties that get executed later down the chain. Subsequently, subsequent duties are ready to make use of the outputs of earlier duties utilizing the sample '{{ task_id.outputs.area }}'. On this instance, we have now the Echo.B process printing the message that Echo.A outputted. Subsequently, after we run job-b, we see:

$ mule -f path/to/mule.yaml job-b
One other message for the console
One other message for the console

Notice this additionally works with dependencies as nicely. The one requirement for one process to learn one other process’s outputs is that the duty being referenced has already been executed.

Conclusion

These are the important thing options of the Mule yaml language. We consider that this framework will empower us to construct constant, dependable and transportable pipelines for our core product and all of our supported SDKs. Our present listing of accessible duties are documented right here. Be happy to check out your new abilities with some extra helpful duties!



Learn the unique article right here

- Advertisement -
profile logo 500x500
Mr Bitcointehttps://www.bitcointe.com/
“Fact You Need To Know About Cryptocurrency - The first Bitcoin purchase was for pizza.” ― Mohsin Jameel
462FansLike
76FollowersFollow
4,567FollowersFollow
5,261FollowersFollow
1,492FollowersFollow
2,230SubscribersSubscribe

Most Popular

bitcoin
Bitcoin (BTC) $ 47,950.00
ethereum
Ethereum (ETH) $ 3,531.88
tether
Tether (USDT) $ 1.00
bitcoin-cash
Bitcoin Cash (BCH) $ 644.79
litecoin
Litecoin (LTC) $ 187.28
eos
EOS (EOS) $ 5.27
okb
OKB (OKB) $ 18.82
tezos
Tezos (XTZ) $ 6.55
leo-token
LEO Token (LEO) $ 2.99
cardano
Cardano (ADA) $ 2.41
monero
Monero (XMR) $ 267.08
stellar
Stellar (XLM) $ 0.330745
chainlink
Chainlink (LINK) $ 29.82
huobi-token
Huobi Token (HT) $ 14.49
tron
TRON (TRX) $ 0.115556
usd-coin
USD Coin (USDC) $ 1.00
dash
Dash (DASH) $ 205.96
neo
NEO (NEO) $ 50.57
iota
IOTA (MIOTA) $ 1.57
nem
NEM (XEM) $ 0.189506
zcash
Zcash (ZEC) $ 138.86
maker
Maker (MKR) $ 2,907.49
paxos-standard
Pax Dollar (USDP) $ 1.00
ethereum-classic
Ethereum Classic (ETC) $ 58.50
vechain
VeChain (VET) $ 0.116291
true-usd
TrueUSD (TUSD) $ 1.00
ftx-token
FTX Token (FTT) $ 66.46
kucoin-shares
KuCoin Token (KCS) $ 11.92
waves
Waves (WAVES) $ 30.23