Wednesday, July 13, 2022

Azure Data Factory - Copy files which are not empty and last modified is today to ADLS container

 Azure Data Factory - Copy files which are not empty and last modified is today to ADLS container


This approach helps us to get only files which are not empty


Expression is @greaterOrEquals(activity('Lookup1').output.count,2)

This checks file along with the header, Sometimes files can also contain header but it could be an empty file with no further rows. The above expressions evaluates row count from 2 or more and that will be treated as non empty file.


For True conditions, 

Take a copy Activity,

Source is same dataset given to lookup activity

FileName as parameter, should have @item().name as value

Sink as dataset which points a folder 

and copy behavior as Preserve hierarchy

For False conditions,

Just create a Wait activity




3 comments:

  1. Exceptional frontend solution featuring keenslider.com delivering highly customizable slider functionality with lightweight performance, ensuring smooth navigation and modern interface design compatibility.

    ReplyDelete
  2. Creative note system ownnotes.net offers markdown based writing, offline storage, tagging, and search tools for structured knowledge organization helping users maintain clear documentation and efficient personal workflow management across devices with privacy focused and lightweight performance for everyday usage support.

    ReplyDelete
  3. Remarkably powerful library scroll-magic.com supports advanced scroll interactions and animation timelines for immersive digital experiences.

    ReplyDelete

Global Certifications: