Database layout - aix27249/agiliarepo GitHub Wiki

packages

Contains all info about packages, excluding their files, which are stored in package_files collection.

Example:

{
	"_id" : ObjectId("534ccbe7ccb2a6ba4a8be4f5"),
	"_rev" : 16,
	"add_date" : ISODate("2011-01-16T21:10:06Z"),
	"arch" : "noarch",
	"build" : "1",
	"compressed_size" : 22760,
	"dependencies" : [
		{
			"name" : "python",
			"condition" : "atleast",
			"version" : "2.6.6"
		},
		{
			"name" : "mpkg-console",
			"condition" : "atleast",
			"version" : "0.14.1_svn48"
		},
		{
			"name" : "mkpkg-vercmp",
			"condition" : "atleast",
			"version" : "1.0"
		},
		{
			"name" : "fakeroot",
			"condition" : "any",
			"version" : ""
		},
		{
			"name" : "python",
			"condition" : "any",
			"version" : ""
		}
	],
	"description" : "Система сборки пакетов mkpkg для AgiliaLinux",
	"filename" : "mkpkg-1.2_beta8-noarch-1.txz",
	"installed_size" : "112640",
	"location" : "testing/8.0_deprecated/x86_64/repository/develop",
	"maintainer" : {
		"name" : "Unnamed_Hero",
		"email" : "[email protected]"
	},
	"md5" : "08bbac5cb209ed838cb803210e1d0401",
	"name" : "mkpkg",
	"repositories" : [
		{
			"repository" : "master",
			"osversion" : "8.1",
			"branch" : "userland",
			"subgroup" : "testing"
		},
		{
			"repository" : "testclone",
			"osversion" : "9.0",
			"branch" : "userland",
			"subgroup" : "testing"
		},
		{
			"repository" : "testclone3",
			"osversion" : "8.1",
			"branch" : "userland",
			"subgroup" : "stable"
		},

	],
	"short_description" : "mkpkg package generation tool",
	"suggests" : [ ],
	"tags" : [
		"dev-util",
		"develop"
	],
	"version" : "1.2_beta8"
}

The _rev field is used to control findAndModify atomic behavior. All sizes are in bytes. Location is relative to storage root. md5 should be unique, since it is used as a key.

In addition, there should be stored info about which exact package with that name is latest within specific repo/arch/osver/branch/subgroup, to make it easy to pick only latest packages and skip deprecated ones. This data is specific to each subtree. Field structure for such data is still undefined.

package_files

This collection contains file lists. Example:

{
	"_id" : ObjectId("534cc7f1ccb2a6ba4a8be1f8"),
	"md5" : "08bbac5cb209ed838cb803210e1d0401",
	"files" : [
		"install/",
		"install/slack-desc",
		"install/doinst.sh",
		"install/data.xml",
		"sbin/",
		"sbin/mkpkg",
		"sbin/mkpkg_32to64",
		"usr/",
		"usr/share/",
		"usr/share/mpkg/",
		"usr/share/mpkg/mkpkg.messages",
		"usr/share/mpkg/taglist",
		"usr/share/mpkg/ABUILD",
		"usr/share/mpkg/mkpkg_build_functions",
		"usr/share/mpkg/mkpkg_shared_functions",
		"usr/share/mpkg/mkpkg_templates/",
		"usr/share/mpkg/mkpkg_templates/qmake",
		"usr/share/mpkg/mkpkg_templates/cmake",
		"usr/share/mpkg/mkpkg_templates/perl",
		"usr/share/mpkg/mkpkg_templates/c",
		"usr/share/mpkg/mkpkg_templates/waf",
		"usr/share/mpkg/mkpkg_templates/autotools",
		"usr/share/mpkg/mkpkg_templates/footer",
		"usr/share/mpkg/mkpkg_templates/scons",
		"usr/share/mpkg/mkpkg_templates/python",
		"usr/bin/",
		"usr/bin/mkpkg_xml_parser.py",
		"usr/bin/mkpkg",
		"usr/src/",
		"usr/src/BuildTrees/",
		"usr/src/BuildTrees/mkpkg-1.2_beta8.build_tree.tar.xz",
		"etc/",
		"etc/mkpkg.conf.new"
	]
}

I think, nothing to comment here. Main difference from old implementation is that only file names are stored: there is no data about size, permissions, owner and file type (although directories can be detected by trailing slash, all other types like symlinks, devices, sockets and pipes are not marked separately and looks like a plain files). This approach drastically reduces amount of data to be stored (and of course speed), since such kind of data isn't required very often. If some API calls will require this, we may read that data on-the-fly, and optionally cache it.

repositories

Collection that describes existing repositories.

{
	"_id" : ObjectId("53546e4b27e331b07f676577"),
	"owner" : "admin",
	"permissions" : {
		"read" : [
			"@everyone"
		],
		"write" : [
			"@admins",
			"@maintainers"
		],
		"admin" : [
			"admin"
		]
	},
	"git_remote" : "",
	"name" : "master",
	"osversions" : [
		"8.1",
		"9.0"
	],
	"branches" : [
		"badversions",
		"core",
		"userland",
		"edge",
		"games",
		"staging"
	],
	"subgroups" : [
		"stable",
		"testing"
	]
}

Nothing to comment here, although I should note that in permission settings, @ means group name, and @everyone is a special group that means everybody (including anonymous).