CloudQuery Product Updates #6

June 10, 2023

Yevgeny Pats
Name
Yevgeny Pats
Twitter
@yevgenypats

Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here!

Apache Arrow

We were heads down adopting Apache Arrow as our in-memory data format!

TLDR:

  • Support for more than 30 types (opens in a new tab) for any source plugin including repeated and nested types like lists, structs, maps.
  • Database source plugins (with CDC) will be able to sync their schema accurately using Apache Arrow type system.
  • Any database that already supports Arrow format natively like ClickHouse, DuckDB will be able to enjoy enhanced performance.

Sources

New Sources

Welcoming 4 new partner and community plugins!

New Official plugins:

Community Contributions to Security Policies:

New Resources

  • AWS: 18 New tables:

    • aws_config_config_rule_compliance_details
    • aws_config_remediation_configurations
    • aws_acmpca_certificate_authorities
    • aws_cloudformation_stack_templates
    • aws_cloudformation_template_summaries
    • aws_cloudwatchlogs_log_group_data_protection_policies
    • aws_config_config_rule_compliance_details
    • aws_config_remediation_configurations
    • aws_config_configuration_aggregators
    • aws_config_delivery_channels
    • aws_config_retention_configurations
    • aws_costexplorer_cost_30d
    • aws_costexplorer_cost_forecast_30d
    • aws_efs_access_points
    • aws_networkfirewall_firewall_policies
    • aws_networkfirewall_rule_groups
    • aws_securityhub_enabled_standards
    • aws_securityhub_hubs

  • Azure: 39 New tables:

    • azure_consumption_billing_*
    • azure_consumption_subscription_*
    • azure_dns_record_sets
    • azure_network_private_link_services
    • azure_network_watcher_flow_logs
    • azure_security_sub_assessments
    • azure_sql_server_advanced_threat_protection_settings
    • azure_sql_server_firewall_rules
    • azure_storage_queue_acl

  • GCP: New Table:

    • gcp_dns_resource_record_sets

  • Datadog: 3 Tables:

    • datadog_rum_events
    • datadog_slo_corrections
    • datadog_slos

  • Digital Ocean: 2 Tables:

    • digitalocean_load_balancers
    • digitalocean_regions

  • GitLab: 2 Tables:

    • gitlab_group_billable_members
    • gitlab_project_members

  • Oracle: 5 Tables:

    • oracle_compute_volume_attachments
    • oracle_virtualnetwork_assigned_public_ips
    • oracle_virtualnetwork_ephemeral_public_ips
    • oracle_virtualnetwork_private_ips
    • oracle_virtualnetwork_reserved_public_ips

  • Snyk: 2 Tables:

    • snyk_groups
    • snyk_group_members

  • Stripe: 2 Tables:

    • stripe_checkout_session_line_items
    • stripe_payment_link_line_items

Policies

Other notable blogs and use-cases

Engineering

Security