:: caropepe.com ::

When is Software Design Complete Enough?

In this age of agile software development, it's too easy to rush through design, and begin coding too soon. In addition to the obvious problems (missed requirements, etc), this can also have devastating effects on the development team when they start seeing that development has built something completely different than what they were expecting.

On the other side of the coin, we've all heard of projects being stuck in "analysis paralysis", that state of never really getting started on a software development project because there is always one more thing to learn and discover. There are a number of reasons for getting hung up like this, but fear of not capturing that last bit of "crucial" information to the success of the project is probably one of the biggest.

We don't want to rush through analysis and begin building too fast, but it's also pretty obvious that we don't want to spend too much time on the design. These competing requirements need to be balanced against each other until an accurate and shared idea of what is going to be built emerges.

So how do we know what design is "complete enough" to get started? The quick answer is when you know what you don't know, and know enough to build what the team expects.

If you think about it, there are relatively few "unknowns" at the beginning of the project. That last project you were part of? It was probably introduced with big, general arm-waving terms, and described with a high-degree of confidence. It sounded relatively easy and everyone was eager to get started, right?

As it progressed, four previously unknown database integrations were "discovered", six new screens had to be added, and the underlying logic was much more complicated than originally thought. Each question answered lead to three new ones. Then you woke up one morning realizing that the simple project described six weeks ago is now a monster that will never get completed at this rate!

Breathe easy. You are now becoming informed...

With time, there is going to come a tipping point; a magical day when, instead of three new questions, there might only be two, or one. You're starting to close in on the solution at this point and there's a light at the end of the tunnel.

Every project goes through this lifecycle during the analysis stage and it should be expected, embraced, and planned for. The phases of the lifecycle probably look something like this:

• Phase 1: Few Unknowns/Un-informed
  
• Phase 2: Many Unknowns/Becoming Informed
  
• Phase 3: Few Unknowns/Informed
  

PLANNING FOR UNKNOWNS

How can you plan for unknowns if you're uninformed? How do you estimate time? These are good questions, and ones you should be asking. The process is pretty straightforward and involves taking an inventory of what you do know, quantifying it, and making educated guesses.

You're probably familiar with the 80/20 rule which (roughly) says, "80% of the results come from 20% of the effort..." During the statement of work, we've got a 30/20 rule (the 80/20 comes later) - a few simple elements (20%) will us a rough idea how involved the project is (with 30% certainty).

Our inventory is part of the statement of work (SOW) and should include simple and obvious elements such as number of screens that interact with the user, the number and relative complexity (low/medium/high) of "critical user tasks", how many tables/columns worth of data appear on each screen, the general number and complexity of any reports that are to be included with the application.

Note that experience with a team, development environment, application functionality & business process help speed up the time and reduce unknowns.

Once you have a handle on quantifying what you know, figuring out man-hours is relatively easy, especially if the tasks are broken down into small enough chunks (which they should be). Determining calendar days is harder, and subject to a large number of variables. However, once you have a multiplier, you can simply multiply the number of man-hours by it, and come up with the number of calendar days required with remarkable accuracy.

APPLICATION DESIGN & ANALYSIS

At some point, once analysis is complete, your design is going to close in on that magical "good enough" where you can get started building the darn thing. But how to know when you're here?

First, it's important to break down the effort into a number of categories. Things like screens & GUI, data integrations, workflow automation & logic, and reporting are all logical breakdowns of a project and often lend themselves to being distinct sections in the design document. These categories should be continuations of the initial tasks used to estimate the project from the statement of work.

Then I use three simple questions to test for determining how complete each section or category is. Remember, we're not talking about a document here; we are interested in understanding. A document should be used to capture information, can be used to demonstrate understanding, but is no substitute for understanding.

All team members should answer the questions for themselves, and as part of the team.

1. Do I feel comfortable that all functionality and business processes used by the customer have been discovered, understood, and documented?

2. Do I understand and agree with all functionality that is to be built into the new application? Are things dispositioned in such a way as to be clear and unambiguous?

3. Is my vision of the completed project the same as, and shared with, all members of the team, including customer representatives?
   

The idea behind these questions is twofold. First, we want to avoid too many surprises further down the development lifecycle. As these same people (analysts, developers, testers & QA, etc) begin interacting with the application, they should be experiencing a design that they are familiar with.

Second, once the design is "complete enough" a whole slew of other tasks can get started. Accurate estimates (with 80% certainty) for the build, test, deploy, and maintenance phases can be made. Test scripts can get developed. And user documentation can be written.

This article was originally published on 2005-05-17 20:10:43, while I was a member the University of Washington School of Medicine IT staff.

Crime in Broadview

SEATTLE -- A while back (probably in December or January) my beater (unlocked) had it's glovebox rifled . Nothing was taken, but the contents were scattered around the interior.

Several neighbors reported having the same thing happen to them over the same weekend.

Then, a few weeks later, I heard that a neighbor up the hill had his house broken into and some items were stolen.

Finally, today I was just forwarded the following email:

Dear Neighbors,

In the past few weeks 7 houses and cars have been burglarized in the whole Broadview area.

The car prowls are occurring between 1:00AM and 4:00AM. Garages have been broken into and tools stolen.

The first cars where thefts occurred were in unlocked cars, but the thieves are now breaking windows to get at items that have a quick resale value.

The items taken from homes likewise are easily sold. Guns have been taken.

These are thieves that are making a living on crime. The police currently have no clues as to who may be doing this.

*Lock your cars. Lock your doors. Set your alarms and do not leave anything of value visible.

Report any incidents and suspicious activity to your neighborhood watch andthe police so we can catch these people soon.

I see that Postacrime.com has a recent entry for a break-in at 125th Street & 1st Ave.

Drop me a note if you have any other information. joe @ caropepe.com (remove the spaces).

-Joe

ARS: Reporting using Mid-Tier and CR 10/CRXI/BOXI on Seperate Server (Windows)

SEATTLE -- Since ARS Mid-Tier 6.3 more BMC Remedy AR System implementations than ever are using web browsers as the client of choice. This version was completely redesigned from earlier efforts and closed the gap in speed and functionality that's long existed between the AR User tool and the Mid-Tier.

While users enjoy the ease of accessing ARS via the web, configuring the Mid-Tier to run Crystal Reports is a bit more complicated.

When the "fat" client (AR User) is installed on your Windows workstation, most people also install the Business Objects Crystal Reports Windows report viewer (it's the default). This makes it simple for AR User to run a report -- it already has a connection to the AR System server API so it's just a matter of grabbing the .rpt template definition file from the Report form, running the ARS query, and handing the results to the Crystal viewer for display.

Architecturally speaking, using a web browser as your client makes this much harder. The browser has no practical [0] way of knowing anything about the AR System, available API's, or what other software is installed on your workstation (if any). The Mid-Tier handles all translation between the HTML your browser understands [1] and the AR System's API.

But what about the reports? This is where the Business Objects/Crystal Reports web viewer comes in. The web viewer is included whenever you install BusinessObjects Enterprise or Crystal Reports Server on a web server and is used to format and display reports over the web. Think of it as the web server's version of the Windows Crystal Reports viewer installed with AR User.

The AR System Mid-Tier simply needs access to the web viewer. Everything else is managed by the AR System [2].

If you have report administrators and developers in your organization be patient explaining how this all works, especially if they are used to hosting all reports on their system. They'll want to manage users, groups, permissions, control who uploads report template definitions (.rpt), and decide what databases your users can access. They need to be made very clear that the AR System handles all of this (be nice -- you may be asking them to install some software on their system...).

Configuring the ARS Mid-Tier to access the web viewer depends on what version of Business Object's software you're running. Crystal Reports 10 is different than Crystal Reports XI, and they're both different than Business Objects XI [3]. With all versions the report server running the web viewer uses the ARS ODBC driver to make the connection back to the ARS server and query for results. In the sections below I'll describe each environment and note any oddities.

Crystal Reports 10 - A URL-based Approach

The URL-based approach is exactly what it sounds like. When you request a report a new browser window is opened with a URL pointing to the web viewer configured in the ARS Report Type form. Hopefully everything (including the report template file) will be where it needs to be (and has the proper permissions) and the web viewer displays the report.

Usually, the record in the Report Type form for "Crystal" includes the $CRTLOC$ parameter for Mid-Tier's Crystal Enterprise 10 Location report server host name, and the $RPTLOC$ parameter for Mid-Tier's Reporting Working Directory. There are additional parameters for login information and the query string (among other things)[4].

In the mean time, the Reporting Working Directory is shared and made accessible to the report server across the network. From the report server's perspective, the Mid-Tier's shared directory is mapped as a local drive (which may require specific permissions) and made accessible to it's web server as a virtual directory (so the web viewer can access it).

CR 10 Key Points:

• ARS ODBC driver installed on report server.
• Mid-tier caches the report template locally in its Reporting Working Directory.
• The Reporting Working Directory must be shared across the network.
• The report server's web server maps a virtual directory to the Mid-Tier's Reporting Working Directory.
• Report template file never actually is stored on the report server.
• Web viewer gets all run-time information via parameters contained in the URL.
  

[DIAGRAM: URL-based Reporting using ARS Mid-Tier & CR10]

Crystal Reports Server XI - An Unmanaged Java-based Approach

The single biggest difference between CR 10 and CRXI (and BOXI) is how the Mid-Tier and the Central Management Server (CMS) communicate with each other. Rather than saving the report template locally and sending the CMS a URL telling it where it's at and what to do with it, the Java-based approach provides direct communication between the Mid-Tier and CMS [5].

Specifically, the Mid-Tier communicates with another bit of BMC Remedy software installed on the report server called the "ARWebReportViewer". This software acts as the go-between for the Mid-Tier and the CMS. It receives the report template, credential information, and query information all in one big package from the Mid-Tier. Then it saves the report template locally to the Reporting Working Directory (on the report server), and hands the rest over to the CMS for processing. Much neater (and faster).

With CRXI the CMS simply processes all the pieces and invokes the web viewer. There is no provision for local caching of this information on the report server (it's not published to the CMS), so you can't access or manipulate it in other ways (like scheduling the report to run overnight, etc.) This is why CRXI is called an "unmanaged" solution.

CRXI Key points:

• ARWebReportViewer and ARS ODBC installed on CRXI report server.
• Report Working Directory located on report server (no network share required).
• Mid-Tier hands off credentials & query to ARWebReportViewer.
• ARWebReportViewer hands off to CMS (for display only).
• Web viewer generates & displays report.
• Report is discarded.

[DIAGRAM: Java-based Reporting using ARS Mid-Tier & CRXI]

BusinessObjects XI - A Managed Java-based Approach

Everything is the same as CRXI, except:

Before a report is displayed it is first published (cached) to a folder of CMS where it remains available for further manipulation making it a "managed" solution. This why during ARWebReportViewer BOXI configuration you specify a CMS Machine Name and CMS Machine Connection Details (CMS folder name - usually "ARReports" - and CMS user credentials) and CRXI configuration does not.

BOXI Key Points:

• ARWebReportViewer and ARS ODBC installed on BOXI server.
• Report Working Directory located on report server (no network share required).
• Mid-Tier hands off credentials & query to ARWebReportViewer
• ARWebReportViewer hands off to CMS (for publishing & display)
• Web viewer generates & displays report.
• Report remains available for further management by BOXI (including scheduling).

[DIAGRAM: Java-based Reporting using ARS Mid-Tier & BOXI]

Common Troubleshooting

• Make sure your Reporting Working Directory is properly configured. If it's not set to the default (say, you have a separate directory for each AR System server running reports), you may have to configure the URL specified in the Report Type form.
• Using the "Properties" context menu of your browser (right-click anywhere on the page) allows you to view the URL of the page you're viewing, even if it isn't enabled by default.
• Permissions are important - Your web server will be running as a particular service account. Make sure this account has permissions to wherever the report templates are stashed.

Final Thoughts

Configuring the AR System for web reporting is actually pretty straightforward, especially once you understand the architecture.

Common problems include not specifying the proper Reporting Working Directory and having the proper permissions setup. You can view the properties and look at the URL in the browser.

[0] Of course tools like ActiveX let a browser interact with the workstation and it's underlying OS, but they are proprietary in nature and are not used in the BMC Remedy environment.

[1] I'm using HTML here in the generic sense, meaning browser-readable content. The ARS Mid-Tier uses a servlet engine (New Atlanta or Tomcat) to render Java server pages (.jsp).

[2] For "unmanaged" installations, anyway. Even with "managed" installations additional involvement is optional. BTW, I often find myself having a simliar conversation with database administrators. They don't typically "get" that the AR System manages itself and insist on medling.

[3] Dont' get me started on the confusion caused by Business Objects and their silly naming conventions.

[4] Is there a handy reference to all of the keywords available in this URL? NOTE: Technically you can configure other report types for different report formats. For example, you could created a report type of CrystalPDF and specify the Adobe PDF format as the output of the report. In theory this should work, but in earlier versions of the Mid-Tier something deep inside was hard-coded to only accept "Crystal" as the report type. You mileage may vary.

[5] The Java-based approach uses Java API's. It is not a pure Java solution.

ITSM Incident Priority & Priority

ITSM Incident Priority & Priority Weight

In the Incident Management configuration, weights are applied to the impact and urgency values (behind the scenes). When a support staff member selects an impact and urgency of an Incident, the Incident Management application adds the numerical weights together in order to calculate the priority weight, then assigns a descriptive priority (such as "Critical").

You can increment/decrement the Priority Weight to provide a more granular level of prioritization. If increasing or decreasing the Priority Weight causes the weight to cross a priority weight threshold to the next level of priority (say, from High to Critical), the Priority field is updated to reflect the new priority.

The default weight values (Impact + Urgency = Priority) include:

Balancing Impact & Urgency

Using independent weight values for both Impact and Urgency allows the support organization to fine-tune how much relative value is placed on the customer's interpretation vs. the support staff's interpretation the request's importance. For example, if the customer reports something as at a low urgency, but IT recognizes it as having a big impact, weighting the impact values accordingly will ensure the final priority recognizes IT's better ability to determine the scope/impact of the problem.

Slowing Down Support Staff

This is something I thought of while working a couple of test tickets yesterday. Using the numeric spinner to increase & decrease priority actually slows down the support staff's ability to change the priority. It isn't trivial spinning that number 20 times (or whatever it is) in order to increase it to the next priority threshold. This has the effect of dampening the ticket's priority, keeping it from bouncing from one extreme to another (perhaps as SLA's are trying to keep up).

Priority Granularity

Having a priority weight allows the support staff to choose the proper impact and urgency, and then make it "more" of the resulting priority. For example, if the resulting weight of a "High" priority ticket is say, 20, then the support staff can make it a "High" / 23, which would indicate to other staff members that it's more important than a standard "High", but doesn't justify an overall priority of "Urgent".

In conclusion, I don't think it's that big of a deal, really. Once the support team becomes familiar with the default values for Impact & Urgency weights and understands that they drive the priority and priority weight. They should also understand that in order to change the priority you can either increase/decrease the priority weight (making adjustments independent of Impact/Urgency), or change the Impact/Urgency values. It's not hard to do.

ARS: Discovery & CMDB

ARS: Discovery & CMDB

There are two primary components of this integration: The BMC Topology Discovery server, and the BMC Atrium CMDB (running on the BMC Remedy Action Request System).

The Topology Discovery server has a number of discovery tasks that scan the defined network segment looking for specific types of devices. Once discovered, these devices, their associated components, and resulting relationships are stored in the Topology Discovery database as a “dataset”.

Multiple discovery tasks can be configured, each looking at different segments, or for different types of devices. Each task can also be scheduled to run automatically, or left to be run manually. There is a single dataset and all discovery tasks populate that dataset.

Once a dataset has been populated with items (a.k.a.; configuration items, or CIs) it can be synchronized with the CMDB. The Topology Discovery synchronization utility is configured with an AR System username/password and uses the ARS API to populate a specific dataset in the CMDB (BMC.TOPOLOGY.IMPORT).

After the BMC.TOPOLOGY.IMPORT dataset is populated with CIs resulting from the discovery, a reconciliation job can be run that merges the temporary topology import dataset with the production dataset (BMC.ASSET).

BEST PRACTICE

As might be expected, the number of configuration items (including their relationships) can easily overwhelm a person(s) designated to manage them. This is particularly true if the degree of discovery detail does not match the degree to which the organization manages devices.

If the organization only manages to the “box” level, swapping out routers, switches, and even servers (per vendor agreements) when a failure occurs, and the discovery tool is configured to the component level (a single server has many components – rack, case, mother board, CPU, memory, hard-drivers, etc. – and discovery can find most of them) there will be a huge amount of unnecessary information stored in the dataset.

What usually happens is both discovery and the CMDB are configured for all devices on all networks (*.*.*.*). It takes 12 hours for the discovery to run, and 4 for the synchronization. Now you’ve got 200K+ items in the CMDB and no idea how to manage them.

This point cannot be understated: Only configure Discovery/CMDB to match the degree of detail that the organization manages devices to.