🥯
Everything
  • Best Practices and Guidelines
    • ChatGPT Prompts
    • Documentation Guideline
    • GitBook Spaces
    • GitHub Guideline
    • Internal Usage
  • Tools
    • End to End Order Flow
      • Order Creation
      • Order Import
      • Order Brokering
      • Order Fulfillment
      • Order Fulfillment Notification
    • Git
      • How to Create Issue on Git
    • Jira
      • Feature Request on Jira
    • Launchpad
      • Fulfillment
        • Stuck Orders in Fulfillment App
      • Job Manager
        • Check Job Frequencies
        • Job Runtime Error
        • Mismatch Time Zone
    • Napita
      • Glossary
      • Schedule Processors
      • View and Manage Processors
      • Flow Definitions
      • Verify Processor Properties
      • Bulletins
      • Data Provenance
      • Troubleshooting
        • Data Export Errors
        • Queue Errors
        • Fetch Put SFTP Retry
    • Netsuite
      • Managing Saved Searches
    • Ofbiz
      • Orders
        • Multicurrency Sales Orders
        • POS Order Refresh Failure
        • Pre-Order Catalog Sync
        • Reindex Order after Solr Down Time
      • Products
        • Product Duplication
      • Inventory
        • Add New Variance Reason
      • Data Manager Configuration
        • Create a Job Using Data Manager Configuration
        • SFTP Issue
    • Shopify
      • How to Access Shopify POS APP
      • How to Create Draft Order in Shopify
    • Tathya
      • Database Configuration
        • Additional Settings
      • Charts Creation
        • Selecting Database & Schema
        • View & Save Chart
        • Charts Access
        • Create Multiple Charts
        • LogInsight Charts
      • Understanding Dashboards
        • Configuring Dashboards
        • Dashboards Access
        • Dashboard Options
        • Dashboard Customizations
        • Link Configuration
        • Set Multi-Day Filters
      • Configuring Alerts & Reports
        • Creating Alerts
        • Creating Reports
        • Alternate Interfaces to Schedule Reports
      • Creating Roles
        • List & Modify Roles
      • Creating User
        • Creating LDAP Account
        • Tathya Authentication
        • Listing User in Tathya
        • User Permissions
      • Troubleshoting
        • Reporting Discrepancies
        • Tathya login Issue
      • How to
        • How to Create User on Tathya
        • How to Create Alpha Role on Tathya
        • How to Request for the Report
  • Client Communication
    • Templated Replies
    • Ticket and Release Mangemnet
  • Monitoring
    • Sanity Checklist
    • System Monitoring Guide
  • Hotwax Deployment & Versions
    • HotWax Commerce Deployment
    • Launch Machine
    • Update OMS Version
    • Rollback OMS Version
    • Optional Plugins
Powered by GitBook
On this page
  • Purpose
  • Configuration for the SFTP Processor
  • 1. Access the SFTP Processor
  • 2. Configuration for Fetch SFTP Processor
  • 3. Configuration for Put SFTP Processor
  • Resolution Steps
  • When the SFTP Processor Starts Working:
  • Monitoring and Verification
  • 1. Check Queued Files for Clients
  • Key Notes
  • Consistency:
  • Regular Monitoring:
Edit on GitHub
Export as PDF
  1. Tools
  2. Napita
  3. Troubleshooting

Fetch Put SFTP Retry

PreviousQueue ErrorsNextNetsuite

Last updated 4 months ago

Purpose

This SOP outlines the steps required to configure and manage SFTP Retry for Fetch SFTP and Put SFTP processors in Apache NiFi, ensuring adherence to best practices. URL:


Configuration for the SFTP Processor

1. Access the SFTP Processor

  • Navigate to Apache NiFi > Processor Group > Fetch/Put SFTP Processor.

2. Configuration for Fetch SFTP Processor

  • Set the comms.failure relationship to Retry. Configure the following values:

    • Number of Retry Attempts: 2

    • Retry Back Off Policy: Penalize

    • Retry Back Off Duration: 10 min (default)

    • Penalty Duration: 30 sec (default)

Add a Funnel:

  • Add a funnel to the Fetch SFTP Processor.

  • Redirect the following relationships to the funnel:

    • comms.failure

    • permission.denied

    • not.found

  • Name the connected relationship: SFTP Fetch Fail.

The relationship name must match exactly

3. Configuration for Put SFTP Processor

  • Set the [failure, reject] relationship to Retry. Configure the following values:

    • Number of Retry Attempts: 2

    • Retry Back Off Policy: Penalize

    • Retry Back Off Duration: 10 min (default)

    • Penalty Duration: 30 sec (default)

Add a Funnel:

  • Add a funnel to the Put SFTP Processor.

  • Redirect the following relationships to the funnel:

    • failure

    • reject

  • Name the connected relationship: SFTP Put Fail.

The relationship name must match exactly

---

Resolution Steps

When the SFTP Processor Starts Working:

Redirect Funnel Relationships:

  1. Access the SFTP processor where the files are queued.

  2. Redirect the funnel relationships (SFTP Fetch Fail or SFTP Put Fail) back to the original processor by connecting the funnel to the respective processor.

    • This will create a loop to re-run the failures.

  3. Process all the queued files.

  4. Perform this action for both the Fetch and Put SFTP processors as applicable

Once the queue has been processed and cleared, remove the connection between the funnel and the original processor to prevent an infinite loop in case of future failures.

Validate:

  • Ensure the queued files are correctly processed after redirection.


Monitoring and Verification

1. Check Queued Files for Clients

Navigate to Summary:

  • Click on the hamburger icon in NiFi's main navigation bar.

  • Select Summary.

Open the Summary Window:

  • A new pop-up window titled "NiFi Summary" will appear.

  • Go to the Connections tab.

Locate Relationships:

  • Search for the relationships "SFTP Fetch Fail" or "SFTP Put Fail" in the list.

  • Select By Name.

  • Sort the Queue (Size) column in descending order by clicking the column header.

Redirect to Processor:

  • Click on the Arrow Icon corresponding to the desired relationship to directly navigate to the associated processor.

  • Review the queued files for the processor and follow the resolution steps mentioned above to ensure proper processing.


Key Notes

Consistency:

  • Ensure all relationship names and funnel configurations strictly adhere to the specified formats:

    • SFTP Fetch Fail

    • SFTP Put Fail

Regular Monitoring:

  • Check for queued files periodically to prevent bottlenecks in data flow.

https://napita.hotwax.io/nifi/