Attachment deduplication

Would be great to have an extension that would help identify duplicate attachments within the same record, or even across records in the same table. I am not sure if anyone else deals with always fighting the storage limits on attachments, but I am sure we have some duplication in there somewhere that could be trimmed back. Thank you!

Hi David, thank you for reaching out to us! We currently have two extensions that could potentially help you with this:

  • Gather Unique Attachments from Airtable Lookup Fields: This extension grabs unique attachments from an attachment or lookup field and copies them to another field. You could run this as part of an automation and then clear the original field afterwards.
  • Compress, Resize or Rotate Images on Airtable: You could use this extension to compress images in attachment fields which would also save you some space!

I hope this helps but please let me know if you have any follow-up questions!

Thank you!

I have been using the compress, resize, rotate extension to get our attachment storage under control. So far I have gotten it from over 400Gb to less than 200Gb, real progress. The only issue is that I have thousands of records to push this through and it started with only doing 10 at at a time. I submitted a ticket and you were able to help get that improved. Now it runs about 60 records before it stops, so better for sure but still takes me running the extension many times per day to catch up on the bulk process.

Once I have it under control I can compress records daily and it will be much cleaner.

I will check our the unique attachment extensions as well, that could help even more.

Thank you!

David Fite

I see, I didn’t realize that was you! :slight_smile:

Each bulk processing run is limited to 10 minutes at the moment. Could this be the limit you’re running into or does it stop before you hit 10 minutes? Either way I’d like to apologize for the inconvenience caused by this!

Have you tried our scheduling option? This could help automate the process for you so the extension would slowly but surely work its way through your base. You can find scheduling options under the advanced menu in the Run section of the extension configuration:

That makes sense, I was not aware of a 10 minute limit on bulk actions, but that is almost certainly what I am experiencing.

I am using the schedule run every hour. Within a few more days I should have it all cleared out with that routing. This is not an urgent need, so that time frame works for me here.

Thank you for your quick response!

David

Ok, I’m glad to hear that this is working for you, even though it’s not quite ideal. I’ll have a chat with our engineers to see if the 10-minute limit per run is something we could raise to make this quicker for you! I’ll report back when I know more.

Sorry again for the inconvenience!