Package com.snowflake.ingest.streaming

Interface SnowflakeStreamingIngestClient

  • All Superinterfaces:
    java.lang.AutoCloseable
    public interface SnowflakeStreamingIngestClient
extends java.lang.AutoCloseable
    A class that is the starting point for using the Streaming Ingest client APIs, a single client maps to exactly one account/database/schema/pipe in Snowflake; however, multiple clients can point to the same account/database/schema/pipe. Each client will contain information for Snowflake authentication and authorization, and it will be used to create one or more `SnowflakeStreamingIngestChannel`

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void close()
      Close the client, which will flush first and then release all the resources
      java.util.concurrent.CompletableFuture<java.lang.Void> close​(boolean waitForFlush, java.time.Duration timeoutDuration)
      Close the client.
      void dropChannel​(java.lang.String channelName)
      Drop the specified channel on the server using the channel name
      ChannelStatusBatch getChannelStatus​(java.util.List<java.lang.String> channelNames)
      Return the channel status for the given list of channels from Snowflake
      java.lang.String getClientName()
      Get the client name
      java.lang.String getDBName()
      Get the database name
      java.util.Map<java.lang.String,​java.lang.String> getLatestCommittedOffsetTokens​(java.util.List<java.lang.String> channelNames)
      Return the latest committed/persisted offset token for the given list of channels in Snowflake
      java.lang.String getPipeName()
      Get the pipe name
      java.lang.String getSchemaName()
      Get the schema name
      void initiateFlush()
      Initiates a flush by the Client which causes all outstanding buffered data to be flushed to Snowflake.
      boolean isClosed()
      Check if the client is closed
      OpenChannelResult openChannel​(java.lang.String channelName)
      Open a channel with a given channel name.
      OpenChannelResult openChannel​(java.lang.String channelName, java.lang.String offsetToken)
      Open a channel with a given channel name.
      java.util.concurrent.CompletableFuture<java.lang.Void> waitForFlush​(java.time.Duration timeoutDuration)
      Asynchronously waits for all buffered data in all channels managed by this client to be flushed to the Snowflake server side.

    • Method Detail

      • close

        void close()
        Close the client, which will flush first and then release all the resources

        TODO (SNOW-1916563): decide whether we want to match the V1 behavior

        Specified by:
        close in interface java.lang.AutoCloseable

      • close

        java.util.concurrent.CompletableFuture<java.lang.Void> close​(boolean waitForFlush,
                                                             @Nonnull
                                                             java.time.Duration timeoutDuration)
        Close the client. If waitForFlush=true this function will make sure all the data in all channels managed by this client is flushed to the Snowflake server side or return a timeout error if the timeout is reached.
        Parameters:
        waitForFlush - whether to wait for all channels to be flushed
        timeoutDuration - timeout duration

      • initiateFlush

        void initiateFlush()
        Initiates a flush by the Client which causes all outstanding buffered data to be flushed to Snowflake. Note that data can still be accepted by the Client - this is an asynchronous call and will return after flush is initiated for all Channels opened by this Client

      • openChannel

        OpenChannelResult openChannel​(java.lang.String channelName)
        Open a channel with a given channel name. The channel is opened on the account/database/schema/pipe defined by the current client. The channel will reuse the latest persisted offset token if the channel is reopened.
        Parameters:
        channelName - the open channel request
        Returns:
        a OpenChannelResult object containing the channel and the channel status

      • openChannel

        OpenChannelResult openChannel​(java.lang.String channelName,
                              java.lang.String offsetToken)
        Open a channel with a given channel name. The channel is opened on the account/database/schema/pipe defined by the current client
        Parameters:
        channelName - the open channel request
        offsetToken - the optional offset token to set on the channel to be opened. If not provided and this reopens an existing channel, the latest persisted offset token will be reused.
        Returns:
        a OpenChannelResult object containing the channel and the channel status

      • dropChannel

        void dropChannel​(java.lang.String channelName)
        Drop the specified channel on the server using the channel name

        Note that this call will blindly drop the latest version of the channel and any pending data will be lost. It will also delete Offset Token and other state from Snowflake servers. So only use it if you are completely done ingesting data for this channel. If you open a channel with the same name in the future, it will behave like a new channel.

        Also see SnowflakeStreamingIngestChannel.close() to close channels. That approach will drop the local version of the channel and if the channel has been concurrently reopened by another client, that version of the channel won't be affected.

        Parameters:
        channelName - the channel name to drop

      • getLatestCommittedOffsetTokens

        java.util.Map<java.lang.String,​java.lang.String> getLatestCommittedOffsetTokens​(java.util.List<java.lang.String> channelNames)
        Return the latest committed/persisted offset token for the given list of channels in Snowflake
        Parameters:
        channelNames - the list of channel names
        Returns:
        the latest committed offset token for the channel

      • getChannelStatus

        ChannelStatusBatch getChannelStatus​(java.util.List<java.lang.String> channelNames)
        Return the channel status for the given list of channels from Snowflake
        Parameters:
        channelNames - the list of channel names
        Returns:
        the channel status for the channel

      • isClosed

        boolean isClosed()
        Check if the client is closed
        Returns:
        a boolean which indicates whether the client is closed

      • waitForFlush

        java.util.concurrent.CompletableFuture<java.lang.Void> waitForFlush​(java.time.Duration timeoutDuration)
        Asynchronously waits for all buffered data in all channels managed by this client to be flushed to the Snowflake server side. This method triggers a flush of all pending data across all channels and waits for the flush operations to complete.

        Behavior:

        • If timeoutDuration is null, the operation will wait indefinitely
        • The method coordinates flush operations across all channels managed by this client
        • If the client is already closed when this method is called, an exception is thrown immediately
        Parameters:
        timeoutDuration - The maximum time to wait for all channels to flush. If null, the operation will wait indefinitely.
        Returns:
        A CompletableFuture<Void> that completes successfully when all channels flush successfully within the timeout period. The future completes exceptionally with:
        • TimeoutException if the timeout was reached before all channels could flush
        • SFException with error code "ClientClosed" if the client is closed
        • Other SFException types if flush operations encounter errors
        Throws:
        java.lang.IllegalArgumentException - if timeoutDuration is negative
        SFException - with error code "ClientClosed" if the client is already closed when called

      • getDBName

        java.lang.String getDBName()
        Get the database name
        Returns:
        name of the database

      • getSchemaName

        java.lang.String getSchemaName()
        Get the schema name
        Returns:
        name of the schema

      • getPipeName

        java.lang.String getPipeName()
        Get the pipe name
        Returns:
        name of the pipe

      • getClientName

        java.lang.String getClientName()
        Get the client name
        Returns:
        the client name