missing root disk - dianaclarke/openstack-notes GitHub Wiki
Reader beware: unlike BDMs, block_device_info does not represent all disks that an instance might have. Significantly, it will not contain any representation of an image-backed local disk, i.e. the root disk of a typical instance which isn't boot-from-volume. Other representations used by the libvirt driver explicitly reconstruct this missing disk. I assume other drivers must do the same. --http://lists.openstack.org/pipermail/openstack-dev/2016-June/097529.html
$ nova list
+--------------------------------------+--------------+--------+------------+-------------+-------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+--------------+--------+------------+-------------+-------------------+
| 8a74d1ae-0e51-43c3-88c4-648842f498ac | instance-foo | ACTIVE | - | Running | private=10.0.0.23 |
+--------------------------------------+--------------+--------+------------+-------------+-------------------+
$ nova flavor-show flavor-foo
+----------------------------+--------------------------------------+
| Property | Value |
+----------------------------+--------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 1 |
| disk | 1 |
| extra_specs | {} |
| id | 8810067f-5199-457d-9b29-b6a2e49f40c6 |
| name | flavor-foo |
| os-flavor-access:is_public | True |
| ram | 111 |
| rxtx_factor | 1.0 |
| swap | 111 |
| vcpus | 1 |
+----------------------------+--------------------------------------+
$ nova show 8a74d1ae-0e51-43c3-88c4-648842f498ac
+--------------------------------------+----------------------------------------------------------------+
| Property | Value |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig | AUTO |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | localhost.localdomain |
| OS-EXT-SRV-ATTR:hostname | instance-foo |
| OS-EXT-SRV-ATTR:hypervisor_hostname | localhost.localdomain |
| OS-EXT-SRV-ATTR:instance_name | instance-00000016 |
| OS-EXT-SRV-ATTR:kernel_id | 08c625c9-e0d8-4197-bf13-e8ec3cbed615 |
| OS-EXT-SRV-ATTR:launch_index | 0 |
| OS-EXT-SRV-ATTR:ramdisk_id | 5e6b021a-2e7d-4193-a0c8-1c6fa298510d |
| OS-EXT-SRV-ATTR:reservation_id | r-7vh17jxv |
| OS-EXT-SRV-ATTR:root_device_name | /dev/vda |
| OS-EXT-SRV-ATTR:user_data | - |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2016-04-22T15:29:26.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | True |
| created | 2016-04-22T15:29:16Z |
| description | instance-foo |
| flavor | flavor-foo (8810067f-5199-457d-9b29-b6a2e49f40c6) |
| hostId | 11f262edbfd66aeac438df805e1c53edfb32d0dc451c05eb767c9dbd |
| host_status | UP |
| id | 8a74d1ae-0e51-43c3-88c4-648842f498ac |
| image | cirros-0.3.4-x86_64-uec (ec3cbb5e-0ce8-4faa-96bd-cfa385122d9a) |
| key_name | - |
| locked | False |
| metadata | {} |
| name | instance-foo |
| os-extended-volumes:volumes_attached | [] |
| private network | 10.0.0.23 |
| progress | 0 |
| security_groups | default |
| status | ACTIVE |
| tenant_id | 162df2e9319041029c8886f07911e9c7 |
| updated | 2016-04-22T15:29:27Z |
| user_id | d83e39dacfc549449c58987c3c99d379 |
+--------------------------------------+----------------------------------------------------------------+
MariaDB [nova]> describe instances;
+--------------------------+-----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------------+-----------------------+------+-----+---------+----------------+
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
| internal_id | int(11) | YES | | NULL | |
| user_id | varchar(255) | YES | | NULL | |
| project_id | varchar(255) | YES | MUL | NULL | |
| image_ref | varchar(255) | YES | | NULL | |
| kernel_id | varchar(255) | YES | | NULL | |
| ramdisk_id | varchar(255) | YES | | NULL | |
| launch_index | int(11) | YES | | NULL | |
| key_name | varchar(255) | YES | | NULL | |
| key_data | mediumtext | YES | | NULL | |
| power_state | int(11) | YES | | NULL | |
| vm_state | varchar(255) | YES | | NULL | |
| memory_mb | int(11) | YES | | NULL | |
| vcpus | int(11) | YES | | NULL | |
| hostname | varchar(255) | YES | | NULL | |
| host | varchar(255) | YES | MUL | NULL | |
| user_data | mediumtext | YES | | NULL | |
| reservation_id | varchar(255) | YES | MUL | NULL | |
| scheduled_at | datetime | YES | | NULL | |
| launched_at | datetime | YES | | NULL | |
| terminated_at | datetime | YES | MUL | NULL | |
| display_name | varchar(255) | YES | | NULL | |
| display_description | varchar(255) | YES | | NULL | |
| availability_zone | varchar(255) | YES | | NULL | |
| locked | tinyint(1) | YES | | NULL | |
| os_type | varchar(255) | YES | | NULL | |
| launched_on | mediumtext | YES | | NULL | |
| instance_type_id | int(11) | YES | | NULL | |
| vm_mode | varchar(255) | YES | | NULL | |
| uuid | varchar(36) | NO | UNI | NULL | |
| architecture | varchar(255) | YES | | NULL | |
| root_device_name | varchar(255) | YES | | NULL | |
| access_ip_v4 | varchar(39) | YES | | NULL | |
| access_ip_v6 | varchar(39) | YES | | NULL | |
| config_drive | varchar(255) | YES | | NULL | |
| task_state | varchar(255) | YES | MUL | NULL | |
| default_ephemeral_device | varchar(255) | YES | | NULL | |
| default_swap_device | varchar(255) | YES | | NULL | |
| progress | int(11) | YES | | NULL | |
| auto_disk_config | tinyint(1) | YES | | NULL | |
| shutdown_terminate | tinyint(1) | YES | | NULL | |
| disable_terminate | tinyint(1) | YES | | NULL | |
| root_gb | int(11) | YES | | NULL | |
| ephemeral_gb | int(11) | YES | | NULL | |
| cell_name | varchar(255) | YES | | NULL | |
| node | varchar(255) | YES | | NULL | |
| deleted | int(11) | YES | MUL | NULL | |
| locked_by | enum('owner','admin') | YES | | NULL | |
| cleaned | int(11) | YES | | NULL | |
| ephemeral_key_uuid | varchar(36) | YES | | NULL | |
+--------------------------+-----------------------+------+-----+---------+----------------+
MariaDB [nova]> describe block_device_mapping;
+-----------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------------+--------------+------+-----+---------+----------------+
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
| device_name | varchar(255) | YES | | NULL | |
| delete_on_termination | tinyint(1) | YES | | NULL | |
| snapshot_id | varchar(36) | YES | MUL | NULL | |
| volume_id | varchar(36) | YES | MUL | NULL | |
| volume_size | int(11) | YES | | NULL | |
| no_device | tinyint(1) | YES | | NULL | |
| connection_info | mediumtext | YES | | NULL | |
| instance_uuid | varchar(36) | YES | MUL | NULL | |
| deleted | int(11) | YES | | NULL | |
| source_type | varchar(255) | YES | | NULL | |
| destination_type | varchar(255) | YES | | NULL | |
| guest_format | varchar(255) | YES | | NULL | |
| device_type | varchar(255) | YES | | NULL | |
| disk_bus | varchar(255) | YES | | NULL | |
| boot_index | int(11) | YES | | NULL | |
| image_id | varchar(36) | YES | | NULL | |
| tag | varchar(255) | YES | | NULL | |
+-----------------------+--------------+------+-----+---------+----------------+
21 rows in set (0.01 sec)
MariaDB [nova]> select * from block_device_mapping where instance_uuid = '8a74d1ae-0e51-43c3-88c4-648842f498ac'\G
*************************** 1. row ***************************
created_at: 2016-04-22 15:29:16
updated_at: 2016-04-22 15:29:18
deleted_at: NULL
id: 64
device_name: /dev/vda
delete_on_termination: 1
snapshot_id: NULL
volume_id: NULL
volume_size: NULL
no_device: 0
connection_info: NULL
instance_uuid: 8a74d1ae-0e51-43c3-88c4-648842f498ac
deleted: 0
source_type: image
destination_type: local
guest_format: NULL
device_type: disk
disk_bus: NULL
boot_index: 0
image_id: ec3cbb5e-0ce8-4faa-96bd-cfa385122d9a
tag: NULL
*************************** 2. row ***************************
created_at: 2016-04-22 15:29:16
updated_at: 2016-04-22 15:29:18
deleted_at: NULL
id: 65
device_name: /dev/vdb
delete_on_termination: 1
snapshot_id: NULL
volume_id: NULL
volume_size: 1
no_device: 0
connection_info: NULL
instance_uuid: 8a74d1ae-0e51-43c3-88c4-648842f498ac
deleted: 0
source_type: blank
destination_type: local
guest_format: NULL
device_type: disk
disk_bus: virtio
boot_index: -1
image_id: NULL
tag: NULL
*************************** 3. row ***************************
created_at: 2016-04-22 15:29:16
updated_at: 2016-04-22 15:29:18
deleted_at: NULL
id: 66
device_name: /dev/vdc
delete_on_termination: 1
snapshot_id: NULL
volume_id: NULL
volume_size: 111
no_device: 0
connection_info: NULL
instance_uuid: 8a74d1ae-0e51-43c3-88c4-648842f498ac
deleted: 0
source_type: blank
destination_type: local
guest_format: swap
device_type: disk
disk_bus: virtio
boot_index: -1
image_id: NULL
tag: NULL
3 rows in set (0.00 sec)
bdms = block_device_obj.block_device_make_list(self.context, [
fake_block_device.FakeDbBlockDeviceDict({
'id': 64,
'device_name': '/dev/vda',
'delete_on_termination': 1,
'snapshot_id': None,
'volume_id': None,
'volume_size': None,
'no_device': 0,
'connection_info': None,
'instance_uuid': '8a74d1ae-0e51-43c3-88c4-648842f498ac',
'deleted': 0,
'source_type': 'image',
'destination_type': 'local',
'guest_format': None,
'device_type': 'disk',
'disk_bus': None,
'boot_index': 0,
'image_id': 'ec3cbb5e-0ce8-4faa-96bd-cfa385122d9a',
'tag': None,
}),
fake_block_device.FakeDbBlockDeviceDict({
'id': 65,
'device_name': '/dev/vdb',
'delete_on_termination': 1,
'snapshot_id': None,
'volume_id': None,
'volume_size': 1,
'no_device': 0,
'connection_info': None,
'instance_uuid': '8a74d1ae-0e51-43c3-88c4-648842f498ac',
'deleted': 0,
'source_type': 'blank',
'destination_type': 'local',
'guest_format': None,
'device_type': 'disk',
'disk_bus': 'virtio',
'boot_index':-1,
'image_id': None,
'tag': None,
}),
fake_block_device.FakeDbBlockDeviceDict({
'id': 66,
'device_name': '/dev/vdc',
'delete_on_termination': 1,
'snapshot_id': None,
'volume_id': None,
'volume_size': 111,
'no_device': 0,
'connection_info': None,
'instance_uuid': '8a74d1ae-0e51-43c3-88c4-648842f498ac',
'deleted': 0,
'source_type': 'blank',
'destination_type': 'local',
'guest_format': 'swap',
'device_type': 'disk',
'disk_bus': 'virtio',
'boot_index':-1,
'image_id': None,
'tag': None,
}),
])
# hmmm... the root disk is indeed missing...
block_device_info = {
'root_device_name': None,
'block_device_mapping': [],
'ephemerals': [{
'device_name': '/dev/vdb',
'num': 0,
'size': 1,
'virtual_name': 'ephemeral0'
}],
'swap': {
'device_name': '/dev/vdc',
'swap_size': 111
}
}
# compared to the existing unit test that looks like so:
block_device_info = {
'swap': None,
'ephemerals': [],
'root_device_name': None,
'block_device_mapping': [{
'connection_info': {'driver_volume_type': 'rbd'},
'mount_device': '/dev/vda',
'delete_on_termination': False,
}],
}
# see: nova.tests.unit.compute.test_compute.ComputeTestCase
# (test_get_instance_block_device_info)
The block_device_mapping info is missing because the root disk lacks connection info.
# if the block_device_mapping has no value in connection_info
# (returned as None), don't include in the mapping
block_device_info['block_device_mapping'] = [
bdm for bdm in driver.block_device_info_get_mapping(
block_device_info)
if bdm.get('connection_info')]
But why isn't it set here?
block_device_info = {
'root_device_name': instance.root_device_name,
'ephemerals': virt_block_device.convert_ephemerals(
block_device_mapping),
'block_device_mapping':
virt_block_device.convert_all_volumes(*block_device_mapping)
}
Because they are _NotTransformable. Why? Because _InvalidType extends _NotTransformable.
if (not self._bdm_obj.source_type == self._valid_source
or not self._bdm_obj.destination_type ==
self._valid_destination):
raise _InvalidType
And this is False:
self._bdm_obj.destination_type == self._valid_destination.self._bdm_obj.destination_type:'local'self._valid_destination:'volume'
class DriverImageBlockDevice(DriverVolumeBlockDevice):
_valid_source = 'image'
_proxy_as_attr = set(['volume_size', 'volume_id', 'image_id'])
def __init__(self, bdm):
super(DriverImageBlockDevice, self).__init__(bdm)
def attach(self, context, instance, volume_api,
virt_driver, wait_func=None, do_check_attach=True):
if not self.volume_id:
av_zone = _get_volume_create_az_value(instance)
vol = volume_api.create(context, self.volume_size,
'', '', image_id=self.image_id,
availability_zone=av_zone)
if wait_func:
self._call_wait_func(context, wait_func, volume_api, vol['id'])
self.volume_id = vol['id']
super(DriverImageBlockDevice, self).attach(
context, instance, volume_api, virt_driver,
do_check_attach=do_check_attach)
These classes are always hardcoded to volume, so perhaps they were never meant for local disks.
class DriverVolumeBlockDevice(DriverBlockDevice):
_valid_destination = 'volume'
It looks like Image block devices were a TODO that just didn't happen.
Another advantage that this refactoring brings is that adding additional block device types, like Image block devices (coming soon in a subsequent patch) that can have destination type of either a hypervisor image or a volume will be just a matter of extending the base class and redefining attach to do the work of creating the volume if needed.
blueprint: improve-block-device-handling
Change-Id: I5b9c3e2d959c602fa22f49db681da918ae0adcea
- So in code like this... if this is the
new_formatwhat's the old format?
def new_format_is_swap(bdm):
if (bdm.get('source_type') == 'blank' and
bdm.get('destination_type') == 'local' and
bdm.get('guest_format') == 'swap'):
return True
return False
Unrelated dump of a instance:
> select * from instances\G
*************************** 1. row ***************************
created_at: 2016-04-15 20:58:59
updated_at: 2016-04-15 20:59:03
deleted_at: NULL
id: 1
internal_id: NULL
user_id: d83e39dacfc549449c58987c3c99d379
project_id: 162df2e9319041029c8886f07911e9c7
image_ref: ec3cbb5e-0ce8-4faa-96bd-cfa385122d9a
kernel_id: 08c625c9-e0d8-4197-bf13-e8ec3cbed615
ramdisk_id: 5e6b021a-2e7d-4193-a0c8-1c6fa298510d
launch_index: 0
key_name: NULL
key_data: NULL
power_state: 0
vm_state: building
memory_mb: 64
vcpus: 1
hostname: instance-1
host: localhost.localdomain
user_data: NULL
reservation_id: r-0e0gnanq
scheduled_at: NULL
launched_at: NULL
terminated_at: NULL
display_name: instance-1
display_description: instance-1
availability_zone: nova
locked: 0
os_type: NULL
launched_on: localhost.localdomain
instance_type_id: 11
vm_mode: NULL
uuid: abb05684-9783-4b1d-b001-868e42a0c310
architecture: NULL
root_device_name: /dev/vda
access_ip_v4: NULL
access_ip_v6: NULL
config_drive:
task_state: spawning
default_ephemeral_device: NULL
default_swap_device: NULL
progress: 0
auto_disk_config: 1
shutdown_terminate: 0
disable_terminate: 0
root_gb: 0
ephemeral_gb: 0
cell_name: NULL
node: localhost.localdomain
deleted: 0
locked_by: NULL
cleaned: 0
ephemeral_key_uuid: NULL
1 row in set (0.00 sec)