Dear All,

Had a problem where a domain would not build recently. This is how I worked through it.

Note ONA is installed in /opt/ona on Red Hat server.

First I created a test_build script and placed this /opt/ona/bin

#!/bin/sh
#
# test_build
# Script to list domains, build a selected domain.
#

ONABASE=`cat /etc/onabase`
ONA_PATH="/var/named/chroot/var/named"
DCM_PATH="${ONABASE}/bin/dcm.pl -l sys_build"
SRV_FQDN=`/bin/hostname -f`

case "$1" in
list)
DOMAIN_LIST=`$DCM_PATH -r build_bind_server_domain_list server=$SRV_FQDN`
for DOMAIN in `echo $DOMAIN_LIST`
do
echo $DOMAIN
done
;;
build)
echo "Building $2"
$DCM_PATH -r build_bind_domain domain=$2
;;
*)
echo $"Usage: $0 {list|build <domain name>}"
exit 1
esac

exit 0

So
cd /opt/ona/bin/
Run test_build list which returned
<domain1>
<domain2>

I then ran test_build build <domain2> which returned the following error
ERROR => Unable to find interface record 490 with name and notes !

Next I got a list of the ONA tables

./dcm.pl -r ona_sql sql="show tables;"

Tables_in_ona
blocks
configuration_types
configurations
contexts
custom_attribute_types
custom_attributes
dcm_module_list
defaults
device_types
devices
dhcp_failover_groups
dhcp_option_entries
dhcp_options
dhcp_pools
dhcp_server_subnets
dns
dns_server_domains
domains
group_assignments
groups
host_roles
hosts
interface_clusters
interfaces
locations
manufacturers
messages
models
permission_assignments
permissions
roles
sequences
sessions
subnet_types
subnets
sys_config
users
vlan_campuses
vlans

As the error mention an interface record. We first find out about this table.
./dcm.pl -r ona_sql sql="desc interfaces;"
Field:Type:Null:Key:Default:Extra
id:int(10) unsigned:NO:PRI::
subnet_id:int(10) unsigned:NO:::
host_id:int(10) unsigned:NO:::
nat_interface_id:int(10) unsigned:NO::0:
ip_addr:int(10) unsigned:NO:::
mac_addr:varchar(12):NO:::
name:varchar(255):NO:::
description:varchar(255):YES:::
last_response:timestamp:YES:::

Now we search based on the id field for 490
./dcm.pl -r ona_sql sql="select * from interfaces where id=490;"
NOTICE => SQL executed successfully - no records returned

So no record exists. This is where the problem is but what DNS record references this. Next look at the dns table structure.
./dcm.pl -r ona_sql sql="desc dns;"
Field:Type:Null:Key:Default:Extra
id:int(10) unsigned:NO:PRI::
domain_id:int(10) unsigned:NO:::
interface_id:int(10) unsigned:NO:::
dns_id:int(10) unsigned:NO::0:
type:varchar(15):NO:::
ttl:int(10) unsigned:NO:::
name:varchar(255):NO:::
ebegin:timestamp:NO::CURRENT_TIMESTAMP:
notes:varchar(128):NO:::
mx_preference:tinyint(5) unsigned:NO:::
txt:varchar(255):NO:::
srv_pri:smallint(5) unsigned:NO:::
srv_weight:smallint(5) unsigned:NO:::
srv_port:smallint(5) unsigned:NO:::

From this we can see that the dns table references interfaces using interface_id. So we look for interface_id = 490 on the dns table.
./dcm.pl -r ona_sql sql="select * from dns where interface_id=490;
id:domain_id:interface_id:dns_id:type:ttl:name:ebegin:notes:mx_preference:txt:srv_pri:srv_weight:srv_port
1016:5:490:998:PTR:0::2010-05-18 10:32:20::0::0:0:0

So 1016 points to interface 490 which does not exist.

But what does 1016 belong to.
Under dns_id we can see another dns_id. If we look up this one
./dcm.pl -r ona_sql sql="select * from dns where id=998;
id:domain_id:interface_id:dns_id:type:ttl:name:ebegin:notes:mx_preference:txt:srv_pri:srv_weight:srv_port
998:1:481:0:A:0:server1:2010-05-18 10:31:01::0::0:0:0

So we can see that the PTR records belong to server1. As can been seen the dns_id=0 means there are no further links.

So now we search to see if there are any other records for 998
./dcm.pl -r ona_sql sql="select * from dns where dns_id=998;
id:domain_id:interface_id:dns_id:type:ttl:name:ebegin:notes:mx_preference:txt:srv_pri:srv_weight:srv_port
999:5:481:998:PTR:0::2010-05-18 09:43:00::0::0:0:0
1016:5:490:998:PTR:0::2010-05-18 10:32:20::0::0:0:0
1017:5:491:998:PTR:0::2010-05-18 10:32:28::0::0:0:0
1024:5:497:998:PTR:0::2010-05-24 14:17:57::0::0:0:0

So we can see there are 4 records that hold pointers for 998. If we do checks on the interfaces table we find that 481 exists, 490, 491 and 497 do not exist. This means 1016,1017 and 1024 need to be removed from the dns table. As you can not use dcm.pl to remove records you will need to use mysql client to connect directly to the database.

mysql -u root -p
<enter password>
connect ona
DELETE FROM dns WHERE id=1016;
DELETE FROM dns WHERE id=1017;
DELETE FROM dns WHERE id=1024;
COMMIT;
Exit;

Now try a test_build again. If you get the build data returned, ie a long list of
1.1.2.10.in-addr.arpa. IN PTR server-44.dummy.com.
2.1.2.10.in-addr.arpa. IN PTR server-45.dummy.com.

Then the build is successful.

Hope this is helpful.

Yours sincerely

David Ruwoldt

druwoldt,

Excellent troubleshooting information.. Thanks for posting it. Now all I need to do is find out why ONA allowed the orphan records and hopefully people wont have to run into what you did.

This is great info for people to understand a bit more about how to use the dcm.pl tool as well as basic structure of the tables.

Dear Matt,

Hit orphaned record again. What happened
1) Create interface that also creates PTR
2) Delete interface but PTR remains (or maybe I do not wait for GUI to update)
3) Select delete on PTR record

This is the second time this has happened to me. So not sure if deleting interface that has associated PTR causes it or my impatience .

Yours sincerely

David Ruwoldt

Great catch.. I have found the bug in the code itself.. now I gotta fix it up. I can't believe I've not run into this one before!..

I'll try and fix it up shortly.. Thanks for the info.