genvid.toolbox.SDK
- class genvid.toolbox.SDK(*args, **kwargs)
Bases:
AWSTool
,ConsulTemplateTool
Common commands for setting up the SDK.
This class wraps the
ClusterAPI
andBastionAPI
for setting up a new Genvid SDK into the current cluster.Changed in version 1.29.0: Moved
load_config_template()
to base classConsulTemplateTool
.Changed in version 1.29.0: Moved
merge_dict()
to base classBaseTool
.Changed in version 1.30.0: Derived from
ProjectTool
instead of itsConsulTemplateTool
ancestor.Changed in version 1.30.0: Removed VaultTool and ConsulTool from the ancestry since they were already part of ConsulTemplateTool.
Changed in version 1.30.0: Moved
load_nomad_job_template()
to base classProjectTool
.Changed in version 1.31.0: Moved
update_config()
to base classProjectTool
.Changed in version 1.33.0: Loaded the binaries paths for a local cluster from
GENVID_SDK_FOLDER/local-services/init
folder, instead of hard coded. Also move the initialization from the static folders fromsetup_jobs()
tosetup()
.Changed in version 1.33.0: Replace
ProjectTool
withConfigurationLoader
, now inheriting fromConsulTemplateTool
instead.Deprecated since version 1.44.0: Moved to new genvid.sdk package.
- NAME = 'sdk'
The name of the logger for this tool. Must be redefined in children.
- SDK_CONFIG_VERSION = '1.7.0'
- cluster_id
The Cluster ID.
- bastionAPI
Interface to the Bastion-API instance.
- clusterAPI
Interface to the cluster Cluster-API instance.
- cluster_vault_secret
Key in the bastion’s Vault where the cluster vault keys are saved.
- GENVID_SDK_SETTINGS_SCHEMA
A mirror of
GENVID_SDK_SETTINGS_SCHEMA
containing the path to the default settings schema.New in version 1.41.0.
- property cloud_provider
- get_remote_cluster_conn()
Return the cluster IP and private key.
- get_remote_cluster_secrets()
Returns the cluster vault initialization secrets.
- setup(bucket: Optional[str] = None, path: str = '/', *, region=None)
Starts the base SDK services.
- Parameters:
bucket – The name of the bucket. Defaults to the return of
get_s3_bucket_id()
.path – The prefix key in the bucket for the images.
- clean()
Stops the base SDK services, removes jobs and clean the config.
- get_sdk_config() dict
Get the default SDK configuration.
The configuration is found either in
GENVID_SDK_FOLDER/local-services/config
if thecluster_id
islocal
, orGENVID_SDK_FOLDER/cloud-services/config
otherwise. The config is loaded and parsed by consul-template first, and the jobs templates are expected to be found in a sibling folder namedtemplates
.Changed in version 1.31.0: Load config from a folder.
- set_config(config: Optional[dict] = None)
Merge the specified configuration with the current one. :param config: The configuration to merge.
- clean_config()
Clean the configuration.
- remove_config(config: dict)
Remove values from the configuration. :param config: The configuration to remove values from.
- static update_dict_no_create(source, dest)
- is_jobs_running()
Return True if a job is configured to be running.
- start_jobs(jobs: Optional[List[str]] = None)
Start all jobs or the specified job. :param jobs: The list of jobs to start.
- stop_jobs(jobs: Optional[List[str]] = None, *, purge: bool = False)
Stop all jobs or the specified job.
- Parameters:
jobs – The list of jobs to stop.
purge – If true, purge the job(s) immediately.
Changed in version 1.45.0: Add the
purge
parameter.
- property default_images_sdk_path
Get default S3 key prefix for SDK images.
New in version 1.20.0.
- upload_images_sdk(bucket=None, path=None, update_config: bool = False, cloud_provider: Optional[str] = None, *, region=None)
Upload the images for the SDK.
- Parameters:
bucket (str) – Bucket ID (default:
get_s3_bucket_id()
).path – Key prefix to use in the bucket.
update_config (bool) – Update cluster images configuration.
region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Default path changes from ‘/’ to ‘/images/sdk-{version}/’.
Changed in version 1.20.0: New
region
parameter following the conventions inAWSTool
.Changed in version 1.26.0:
Added parameter
region
to target another region for the S3 bucket where the image will be uploaded.If the bucket, named after the Bastion Id, does not exist, it will be created in the targeted
region
.If it already exists but in a different region, the underlying code will raise an exception.
By default, the region used to create the bucket will be the region of the current AWS profile.
It can also be overriden by the AWS_DEFAULT_REGION environment variable.
If the
region
parameter is None and the bucket region already exists but its region differs from the one of the profile or AWS_DEFAULT_REGION environment variable, it will simply be used.
- get_s3_bucket_id()
Get the S3 bucket ID for images.
Note
This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.
- get_s3_bucket_with_bastion_id(bastion_id: str)
Get the S3 bucket ID for images using the current AWS account and the Bastion ID.
Note
This method makes a global STS request to AWS. We disallow setting the region for this call to avoid confusion with the region of the bucket itself.
- update_images_config(bucket: Optional[str] = None, path: str = '/images/custom', *, delete: bool = False, region: Optional[str] = None)
Update the images configuration on the cluster.
- Parameters:
bucket – Bucket ID (default:
get_s3_bucket_id()
).path – Key prefix to use in the bucket.
delete – Do not update configuration, delete it instead.
region – Region containing the S3 bucket.
Warning
This method can only be called on a non-local cluster.
New in version 1.20.0.
- upload_images(prefixes: List[str], bucket: Optional[str] = None, path: str = '/images/custom', update_config: bool = False, basedir: Optional[Union[Path, str]] = None, cloud_provider: Optional[str] = None, *, region: Optional[str] = None)
Upload the images with the specified prefixes.
Note
The image names must match
DockerTool.RE_IMAGE_IMAGEID
.- Parameters:
prefixes – Only upload the files with these prefixes.
bucket (str) – Bucket ID (default:
get_s3_bucket_id()
).path – Key prefix to use in the bucket.
update_config (bool) – Update cluster images configuration.
basedir – Source directory (default:
GENVID_SDK_FOLDER/images
).region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.
Changed in version 1.20.0:
Added the
basedir
parameter.Augmented error handling.
New
region
parameter following the conventions inAWSTool
.Factored the
update_config
part of the method into its own method.
Changed in version 1.26.0:
Added parameter
region
to target another region for the S3 bucket where the image will be uploaded.If the bucket, named after the Bastion Id, does not exist, it will be created in the targeted
region
.If it already exists but in a different region, the underlying code will raise an exception.
By default, the region used to create the bucket will be the region of the current AWS profile.
It can also be overriden by the AWS_DEFAULT_REGION environment variable.
If the
region
parameter is None and the bucket region already exists but its region differs from the one of the profile or AWS_DEFAULT_REGION environment variable, it will simply be used.
- create_nats_window_package()
Creates
nats-window
package if it does not exist.Changed in version 1.32.0: Make it run on Linux using the binaries file.
- get_images_config(bucket: Optional[str] = None, path: str = '/images/custom', *, region: Optional[str] = None) dict
- Retrieve images configurations from an S3 bucket or azure container
based on cloud provider.
- Parameters:
bucket – Bucket ID .
path – Key prefix in the bucket (default: ‘/images/custom’).
region (str) – Region containing the S3 bucket/Azure Container.
Important
The image names must match
DockerTool.RE_IMAGE_IMAGEID
to be selected.- Returns:
Images configurations as a
dict
.
Changed in version 1.14.0: Changed the default path from ‘/’ to ‘/images/custom’.
Changed in version 1.20.0: New
region
parameter following the conventions inAWSTool
.
- clean_images(bucket: Optional[str] = None, path: str = '/images/custom', update_config: bool = False, *, region: Optional[str] = None)
Remove all images from an S3 bucket.
- Parameters:
bucket – Bucket ID (default:
get_s3_bucket_id()
).path – Key prefix to use in the bucket.
update_config (bool) – Update cluster images configuration.
region (str) – Region containing the S3 bucket.
Warning
update_config
can only be set for non-local clusters.Changed in version 1.14.0: Change the default path from ‘/’ to ‘/images/custom’.
Changed in version 1.20.0:
New
region
parameter following the conventions inAWSTool
.Now delegates to
update_images_config()
forupdate_config
.
- update_archive(archive: Union[Path, str], version: str = 'dev', destdir: Optional[Union[Path, str]] = None, clean: bool = True, keep: bool = False)
Update a binary archive to conform to the image format expected.
- Parameters:
archive – The path to the archive file. The filename should always be in the form of
name.archfmt
.version – Version of the archive.
destdir – Destination directory. Default to the same as the original archive.
clean – Clean old versions of the archive (the ones with the same name).
keep – Keep the original archives. You should almost always used it with a different
destdir
.
New in version 1.20.0.
Deprecated since version 1.30.0: Method moved to
PackageArchiver
- load_config(source: Union[Path, str], *, job_template_dir: Optional[Union[Path, str]], with_consul_template: bool, dryrun: bool = False) dict
Loads the configuration file(s) (all files in a directory or a specific file) from the source.
- Parameters:
source – Path to the configuration file/folder.
with_consul_template – Execute consul-template on the configuration file(s) (if source is a directory) before using it only if it is set to true.
job_template_dir – Path to Folder from where nomad templates should be loaded. If not mentioned, no jobs will be loaded.
dryrun – If True, only parse the source and returns the configuration.
- Returns:
The configuration as a dictionary that can be used directly with
set_config()
.
- :raises FileNotFoundErrorIf the job_template_dir is not None and
is not a valid directory.
Changed in version 1.26.0: Fix support for
pathlib.Path
for arguments. Add load parameterChanged in version 1.30.0: Refactored to use
load_config_folder()
instead of creating a new instance ofProjectTool
.Changed in version 1.33.0: Use the new render_template command instead of the local consul-template.
- get_output_public_ips() list
Retrieves all public ip’s from output of terraform module.
- Returns:
List of all public IP Addressses
- get_cloud_provider() str
Retrieves the cloud provider from output of terraform module.
If no cloud provider is found in terraform’s cluster output, aws is returned as default.
- Returns:
Cloud provider.
- load_config_template(template: Union[Path, str], env: Optional[Mapping[str, str]] = None) Any
Deprecated since version 1.33.0:
ConsulTemplateTool
will no longer be part of the SDK ancestors. Userender_template()
instead.
- consul_template_once(*, use_consul: bool = False, use_vault: bool = False, env: Optional[Mapping[str, str]] = None, **kwargs) ConsulTemplateOutput
Deprecated since version 1.33.0:
ConsulTemplateTool
will no longer be part of the SDK ancestors. Userender_template()
instead.
- property consul_template
Deprecated since version 1.33.0:
ConsulTemplateTool
will no longer be part of the SDK ancestors. Userender_template()
instead.
- class sdk.SDK
Implementation of
genvid.toolbox.SDK