How can I prevent apt-mirror from downloading ALL packages

up vote
1
down vote

favorite

I'm trying to configure apt-mirror, it's working fine for ubuntu upstreams but when I use it for something like GitLab, it downloads all package versions.

For example, gitlab-ce is 72GB, each version is under 300MB. How can I keep just the latest or even the last 1 or 2 packages, not 459?

edited May 17 at 17:45

Martin Thornton

2,39541730

asked May 8 at 16:51

Jacob Evans

585

add a commentÂ |Â

up vote
1
down vote

favorite

I'm trying to configure apt-mirror, it's working fine for ubuntu upstreams but when I use it for something like GitLab, it downloads all package versions.

For example, gitlab-ce is 72GB, each version is under 300MB. How can I keep just the latest or even the last 1 or 2 packages, not 459?

edited May 17 at 17:45

Martin Thornton

2,39541730

asked May 8 at 16:51

Jacob Evans

585

add a commentÂ |Â

up vote
1
down vote

favorite

I'm trying to configure apt-mirror, it's working fine for ubuntu upstreams but when I use it for something like GitLab, it downloads all package versions.

For example, gitlab-ce is 72GB, each version is under 300MB. How can I keep just the latest or even the last 1 or 2 packages, not 459?

edited May 17 at 17:45

Martin Thornton

2,39541730

asked May 8 at 16:51

Jacob Evans

585

I'm trying to configure apt-mirror, it's working fine for ubuntu upstreams but when I use it for something like GitLab, it downloads all package versions.

For example, gitlab-ce is 72GB, each version is under 300MB. How can I keep just the latest or even the last 1 or 2 packages, not 459?

edited May 17 at 17:45

Martin Thornton

2,39541730

asked May 8 at 16:51

Jacob Evans

585

edited May 17 at 17:45

Martin Thornton

2,39541730

edited May 17 at 17:45

Martin Thornton

2,39541730

edited May 17 at 17:45

Martin Thornton

2,39541730

asked May 8 at 16:51

Jacob Evans

585

asked May 8 at 16:51

Jacob Evans

585

asked May 8 at 16:51

Jacob Evans

585

add a commentÂ |Â

1 Answer
1

active

oldest

votes

up vote
2
down vote

accepted

+50

APT Mirror is intended to mirror the whole repository from mirror.list entries only.

In a few sentences apt-mirror works like this:

Gets index files from the repository and processes them (the same process as apt update).

Downloads files from collected index files, if they do not exist locally. The folder structure is preserved in this process (similar to wget -c -x <url_file_array>).

For more detailed information about how apt-mirror works and for confirming that it is impossible to partially mirror the repository to get latest versions only, you can refer to The Source code of apt-mirror (Written in Perl).

So, what to do to get latest versions from large repositories?

My temporary workaround for your situation (taking for example, gitlab-ce repository for Ubuntu 18.04 - Bionic):

Enter the path for mirroring:
```
cd /path/to/mirroring
```

Backup sources.list:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Open and comment every line in sources.list:
```
sudo nano /etc/apt/sources.list
```

Add your desired repository(ies) for mirroring:

4.1. Add this to sources.list and exit from the file: deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ bionic main

4.2. Add the GPG:
```
curl -L https://packages.gitlab.com/gitlab/gitlab-ce/gpgkey | sudo apt-key add -
```

Update index files:
```
sudo apt update
```

Fetch latest packages url from added repository and write them to file:

sudo apt-get install '*' --allow-unauthenticated -y --print-uris | grep -o ''http.*' | tr "'" " " > download-list

Download the urls with folder structure, because of -c option, it will not download files that exist:
```
wget -i download-list -c -x
```
Enjoy!!!

For reverting everything back, just replace the sources.list file with old sources.list.bak and do:
```
sudo apt update
```

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

add a commentÂ |Â

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "89"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');

var $window = $(window),
onScroll = function(e)
var $elem = $('.new-login-left'),
docViewTop = $window.scrollTop(),
docViewBottom = docViewTop + $window.height(),
elemTop = $elem.offset().top,
elemBottom = elemTop + $elem.height();
if ((docViewTop elemBottom))
StackExchange.using('gps', function() StackExchange.gps.track('embedded_signup_form.view', location: 'question_page' ); );
$window.unbind('scroll', onScroll);

;
$window.on('scroll', onScroll);

);

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1033669%2fhow-can-i-prevent-apt-mirror-from-downloading-all-packages%23new-answer', 'question_page');

);

Post as a guest

Name

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
2
down vote

accepted

+50

APT Mirror is intended to mirror the whole repository from mirror.list entries only.

In a few sentences apt-mirror works like this:

Gets index files from the repository and processes them (the same process as apt update).

Downloads files from collected index files, if they do not exist locally. The folder structure is preserved in this process (similar to wget -c -x <url_file_array>).

So, what to do to get latest versions from large repositories?

My temporary workaround for your situation (taking for example, gitlab-ce repository for Ubuntu 18.04 - Bionic):

Enter the path for mirroring:
```
cd /path/to/mirroring
```

Backup sources.list:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Open and comment every line in sources.list:
```
sudo nano /etc/apt/sources.list
```

Add your desired repository(ies) for mirroring:

4.1. Add this to sources.list and exit from the file: deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ bionic main

4.2. Add the GPG:
```
curl -L https://packages.gitlab.com/gitlab/gitlab-ce/gpgkey | sudo apt-key add -
```

Update index files:
```
sudo apt update
```

Fetch latest packages url from added repository and write them to file:

sudo apt-get install '*' --allow-unauthenticated -y --print-uris | grep -o ''http.*' | tr "'" " " > download-list

Download the urls with folder structure, because of -c option, it will not download files that exist:
```
wget -i download-list -c -x
```
Enjoy!!!

For reverting everything back, just replace the sources.list file with old sources.list.bak and do:
```
sudo apt update
```

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

add a commentÂ |Â

up vote
2
down vote

accepted

+50

APT Mirror is intended to mirror the whole repository from mirror.list entries only.

In a few sentences apt-mirror works like this:

Gets index files from the repository and processes them (the same process as apt update).

Downloads files from collected index files, if they do not exist locally. The folder structure is preserved in this process (similar to wget -c -x <url_file_array>).

So, what to do to get latest versions from large repositories?

My temporary workaround for your situation (taking for example, gitlab-ce repository for Ubuntu 18.04 - Bionic):

Enter the path for mirroring:
```
cd /path/to/mirroring
```

Backup sources.list:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Open and comment every line in sources.list:
```
sudo nano /etc/apt/sources.list
```

Add your desired repository(ies) for mirroring:

4.1. Add this to sources.list and exit from the file: deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ bionic main

4.2. Add the GPG:
```
curl -L https://packages.gitlab.com/gitlab/gitlab-ce/gpgkey | sudo apt-key add -
```

Update index files:
```
sudo apt update
```

Fetch latest packages url from added repository and write them to file:

sudo apt-get install '*' --allow-unauthenticated -y --print-uris | grep -o ''http.*' | tr "'" " " > download-list

Download the urls with folder structure, because of -c option, it will not download files that exist:
```
wget -i download-list -c -x
```
Enjoy!!!

For reverting everything back, just replace the sources.list file with old sources.list.bak and do:
```
sudo apt update
```

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

add a commentÂ |Â

up vote
2
down vote

accepted

+50

up vote
2
down vote

accepted

+50

APT Mirror is intended to mirror the whole repository from mirror.list entries only.

In a few sentences apt-mirror works like this:

Gets index files from the repository and processes them (the same process as apt update).

Downloads files from collected index files, if they do not exist locally. The folder structure is preserved in this process (similar to wget -c -x <url_file_array>).

So, what to do to get latest versions from large repositories?

My temporary workaround for your situation (taking for example, gitlab-ce repository for Ubuntu 18.04 - Bionic):

Enter the path for mirroring:
```
cd /path/to/mirroring
```

Backup sources.list:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Open and comment every line in sources.list:
```
sudo nano /etc/apt/sources.list
```

Add your desired repository(ies) for mirroring:

4.1. Add this to sources.list and exit from the file: deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ bionic main

4.2. Add the GPG:
```
curl -L https://packages.gitlab.com/gitlab/gitlab-ce/gpgkey | sudo apt-key add -
```

Update index files:
```
sudo apt update
```

Fetch latest packages url from added repository and write them to file:

sudo apt-get install '*' --allow-unauthenticated -y --print-uris | grep -o ''http.*' | tr "'" " " > download-list

Download the urls with folder structure, because of -c option, it will not download files that exist:
```
wget -i download-list -c -x
```
Enjoy!!!

For reverting everything back, just replace the sources.list file with old sources.list.bak and do:
```
sudo apt update
```

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

APT Mirror is intended to mirror the whole repository from mirror.list entries only.

In a few sentences apt-mirror works like this:

Gets index files from the repository and processes them (the same process as apt update).

Downloads files from collected index files, if they do not exist locally. The folder structure is preserved in this process (similar to wget -c -x <url_file_array>).

So, what to do to get latest versions from large repositories?

My temporary workaround for your situation (taking for example, gitlab-ce repository for Ubuntu 18.04 - Bionic):

Enter the path for mirroring:
```
cd /path/to/mirroring
```

Backup sources.list:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak

Open and comment every line in sources.list:
```
sudo nano /etc/apt/sources.list
```

Add your desired repository(ies) for mirroring:

4.1. Add this to sources.list and exit from the file: deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu/ bionic main

4.2. Add the GPG:
```
curl -L https://packages.gitlab.com/gitlab/gitlab-ce/gpgkey | sudo apt-key add -
```

Update index files:
```
sudo apt update
```

Fetch latest packages url from added repository and write them to file:

sudo apt-get install '*' --allow-unauthenticated -y --print-uris | grep -o ''http.*' | tr "'" " " > download-list

Download the urls with folder structure, because of -c option, it will not download files that exist:
```
wget -i download-list -c -x
```
Enjoy!!!

For reverting everything back, just replace the sources.list file with old sources.list.bak and do:
```
sudo apt update
```

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

edited May 18 at 19:21

answered May 11 at 16:48

Olimjon

1,872423

answered May 11 at 16:48

Olimjon

1,872423

answered May 11 at 16:48

Olimjon

1,872423

add a commentÂ |Â

draft saved

draft discarded

draft saved

draft discarded

Sign up or log in

Post as a guest

Name

Sign up or log in

Name

搜尋此網誌

Gfilui