Zach Burlingame

Programming, Computers, and Other Notes on Technology

HOWTO: Create a Local Repository Mirror on Ubuntu

How to create and use a local repository mirror on Ubuntu 9.10. These instructions should work with minor modifications for other versions of Ubuntu.

# File: HOWTO Create a Local Repository Mirror on Ubuntu.notes
# Date: 2010/03/17
# Refs: https://help.ubuntu.com/community/Debmirror
#       http://ubuntuforums.org/archive/index.php/t-599479.html
#       http://www.arsgeek.com/2007/02/14/how-to-set-up-your-own-local-repositories-with-apt-mirror/
#       http://pwet.fr/man/linux/commandes/debmirror
# Desc: How to create a local repository for 
#       Ubuntu 9.10 Karmic Koala.

# -------------------------------------
#           Setup the Server
# -------------------------------------
# Install Ubuntu (I used 9.10) on a machine with plenty of 
# free storage space (I used an 8GB OS vmdk and an 80GB data 
# vmdk used through LVM so that I could easily add/grow to
# it in the future if necessary).

# Create the mirror user, I'll be using ubuntu.
# NOTE: You don't have to add this user to the wheel but if you don't, the steps below that require sudo
#       will require you to run them from an account with root or wheel access and may also require
#       that you change the ownership/group of files/directories afterwards.
sudo useradd -m ubuntu -Gusers,wheel
sudo password ubuntu

# UPDATE 2012/01/30: As Dave points out below, you'll need to create your mirrorkeyring folder with the correct user account.
#                    If you aren't already running as that user, you can change your shell using su
su - ubuntu

# Update your apt-get package listing
sudo apt-get update

# Install debmirror
sudo apt-get install debmirror

# Create the location for the repo data to live
sudo mkdir -P /mirror/ubuntu

# Set the permissions for the repo data
sudo chown -R ubuntu:ubuntu /mirror/ubuntu
sudo chmod -R 771 /mirror/ubuntu

# Setup the keyring for correctly verifying Release signatures
gpg --no-default-keyring --keyring /home/ubuntu/mirrorkeyring/trustedkeys.gpg --import /usr/share/keyrings/ubuntu-archive-keyring.gpg

# Create the mirrorbuild.sh script
vim /home/ubuntu/mirrorbuild.sh

# NOTE: The ubuntu community documentation has you using 
#       the HTTP protocol for the mirror build script
#       however, I prefer rsync because we can rate limit.
#       When the download is going to take days,
#       I'd like to be able to use my connection in
#       the interim.

# --------------------------------------------
# BEGIN MIRRORBUILD.SH SCRIPT
# --------------------------------------------

#!/bin/bash

## Setting variables with explanations.

#
# Don't touch the user's keyring, have our own instead
#
export GNUPGHOME=/home/ubuntu/mirrorkeyring

# Arch=         -a      # Architecture. 
# For Ubuntu can be i386, amd64, powerpc and/or sparc (sparc support begins with dapper)
# 
# Comma separated values
arch=i386,amd64

# Minimum Ubuntu system requires main, restricted
# Section=      -s      # Section
# (One of the following - main/restricted/universe/multiverse).
# You can add extra file with $Section/debian-installer.
# ex: main/debian-installer,universe/debian-installer,multiverse/debian-installer,restricted/debian-installer
section=main,restricted,universe,multiverse

# Release=      -d      # Release of the system
# (Dapper, Edgy, Feisty, Gutsy, Hardy, IntrepidJaunty, Karmic), 
# and the -updates and -security ( -backports can be added if desired)
dist=karmic,karmic-updates,karmic-security

# Server=       -h      # Server name,
# minus the protocol and the path at the end
# CHANGE "*" to equal the mirror you want to create your
# mirror from. au. in Australia  ca. in Canada. This can be 
# found in your own /etc/apt/sources.list file, 
# assuming you have Ubuntu installed.
server=us.archive.ubuntu.com

# Dir=          -r      # Path from the main server,
# so http://my.web.server/$dir, Server dependant
# Lead with a '/' for everything but rsync,
# where we lead with a ':'
inPath=:ubuntu

# Proto=        -e      # Protocol to use for transfer
# (http, ftp, hftp, rsync)
# Choose one - http is most usual the service, and the
# service must be availabee on the server you point at.
# NOTE: debmirror uses -aIL --partial by default.
#       However, if you provide the --rsync-options
#       paramter (which we do) then you HAVE to provide 
#       it -aIL --partial in addition to whatever You
#       want to add (e.g. --bwlimit) If you don't
#       debmirror will exit with thousands of files
#       missing.
proto=rsync
rsyncoptions="-aIL --partial --bwlimit=100"

# Outpath=              # Directory to store the mirror in
# Make this a full path to where you want to mirror the material.
#
outPath=/mirror/ubuntu/

# The --nosource option only downloads debs and not deb-src's
# The --progress option shows files as they are downloaded
# --source \ in the place of --no-source \ if you want sources also.
# --nocleanup  Do not clean up the local mirror after mirroring
# is complete. Use this option to keep older repository
# Start script
#
debmirror       -a $arch \
                --no-source \
                -s $section \
                -h $server \
                -d $dist \
                -r $inPath \
                --progress \
                -e $proto \
                --rsync-options="$rsyncoptions" \
                $outPath

# -----------------------------------------------------
# END BUILDMIRROR.SH SCRIPT 
# -----------------------------------------------------

# Add execute permissions on the mirrorbuild.sh script
chmod +x mirrorbuild.sh

# Run the script
./mirrorbuild.sh

# Go home, kick back, have a beer while it downloads 43GBs 
# (in the case of karmic, karmic-update, karmic-securty for
# i386 and amd64)

# --------------------------------------
#          Setup the mirror
# --------------------------------------
# Install apache2
sudo apt-get install apache2

# Symlink the mirror data into the web root
sudo ln -s /mirror/ubuntu /var/www/ubuntu

# Point your browser at http://localhost/ubuntu and
# you should see your pool!

# -------------------------------------
#        Updating the Repo Mirror
# -------------------------------------
# To update the repo mirror, just execute the mirrorbuild.sh
# script used to initially build it.
./mirrorbuild.sh

# -------------------------------------
#   Configure Clients to Use this Repo
# -------------------------------------
# Update the apt sources list
cd /etc/apt
sudo mv sources.list sources.list.orig
sudo sensible-editor sources.list

# Replace 'mirrorbox' with your server's DNS name 
# (e.g. karmic-repo.test.com)
# -----------------------------------------------------------------------------
# BEGIN SOURCES.LIST
# -----------------------------------------------------------------------------
# Local network mirror sources.
deb http://mirrorbox/ubuntu karmic main restricted universe multiverse
deb http://mirrorbox/ubuntu karmic-updates main restricted universe multiverse
deb http://mirrorbox/ubuntu karmic-security main restricted universe multiverse
# -----------------------------------------------------------------------------
# END SOURCES.LIST
# -----------------------------------------------------------------------------

# Test to see if you are able to pull down updates 
# from the new mirror
sudo apt-get update
Tags:

4 Responses to “HOWTO: Create a Local Repository Mirror on Ubuntu”

  • Dave Miles says:

    Hi

    Tried this out on 10.04, using http from behind a proxy since our company policy doesn’t allow rsync connections. Couple of comments, you do need to create the mirrorkeyring folder as the correct user or it doesn’t work, probably worth mentioning in the scripts to create the Ubuntu user first, cos this tripped me up and I don’t consider myself a stupid person!. Added versions up to 12.04 (Precise) and its happily getting 10.04 (which we use in production) and 12.04 (for testing)

    Great work…..

    Dave

    • ZachB says:

      Thanks for pointing that out. I’ve updated the howto to mention creating the user and what happens if you don’t add them to the wheel. Glad you found it helpful!

  • Hi, Zach.

    If i have to create a repository, of maverick meerkat i only have to change karmic to maverick?

    Great work!!!!

  • Yohan says:

    Hi Zach,

    First of all.. simply EXCELLENT tutorial..
    I searched all over the net for something simple and easy to understand.. almost gave up the effort.. but then bumped into your site..
    Now i have a Ubuntu 12.04 local repository running in my Lab..

    Simple.. efficient.. and easy tutorial..

    Keep up the good work.
    Yohan.
    http://yoyoclouds.com

  • Leave a Reply

    *