Ticket #945 (closed enhancement: fixed)

Opened 6 months ago

Last modified 2 weeks ago

support article packages >4GB in size

Reported by: russ Assigned to: pradeep
Priority: critical Milestone: 0.9.1
Component: ambra Version: 0.9-SNAPSHOT
Keywords: Cc:
Blocking: Blocked By:

Description

the zip format is limited to 4GB max filesize.

some articles are larger than 4GB.

we should support other archive formats (i vote for tar+gz) to handle this.

this is a requirement for the plos bio/med ambra migration as we have on giant article that needs to be ingested.

Dependency Graph

Change History

05/28/08 10:02:59 changed by amit

I have found a package in public domain that handles tar files and have added it to our maven repository as <com.ice>.<tar>. Please see:

http://www.gjt.org/javadoc/com/ice/tar/package-summary.html

09/08/08 15:35:13 changed by amit

  • milestone set to 0.9.1.

09/08/08 16:01:44 changed by amit

  • owner changed from amit to ronald.
  • priority changed from high to critical.

10/02/08 11:41:53 changed by amit

This might be a better option Apache Commons Compress as it provides a common abstraction on top of zip, tar and bzip. I was just starting to outline the abstraction when a search on Google popped this on the screen. I have sent an email to the main developer listed on the project inquiring on the status.

10/27/08 13:54:26 changed by amit

  • owner changed from ronald to pradeep.

11/03/08 19:32:37 changed by pradeep

(In [6637]) Support tar, tgz, tbz formats for ingest using commons archiver. Performance sucks for tgz and tbz. tar is just fine.

Addresses #945.

11/03/08 19:32:41 changed by pradeep

  • status changed from new to closed.
  • resolution set to fixed.

(In [6638]) Speed up tgz, tbz ingests by caching entries in java.io.tmpdir.

Closes #945.

11/06/08 16:13:28 changed by russ

  • status changed from closed to reopened.
  • resolution deleted.

prepare-sip should be updated to work with tar/tgz as well ;(

11/10/08 09:54:38 changed by pradeep

  • status changed from reopened to closed.
  • resolution set to fixed.

(In [6642]) Support tar files and compressed tars(gzip, bzip2) also in prepare-sip. Note that the file-type is guessed from the archive file name extension. .zip, .tar, .tgz, .tbz, .tar.bz2, .tar.gz etc. are all valid extensions for an archive file.

Closes #945.