Concrete Thoughts …

Getting rid of infrastructure and fomenting digital transformation

When we migrate business software like CRM, Accounting, Fundraising, Student Information, Collections Management, Content Management, or anything else to the Cloud or to SaaS (which is probably on some Cloud–so I’m going to use the terms interchangeably in this post), we accrue a lot of benefits. Remote access is easier, and you don’t need any special client software to be installed, just a browser, so your staff can get access on any computer nearby, and probably on their phones, too. Another less commonly understood benefit is the savings in labor and specialization-related costs.

What do I mean? In basic terms, you’re trading ownership, management, and maintenance of system software and on-premise infrastructure for a monthly fee. That monthly fee is probably high compared to what you’re used to–SaaS versions of business systems are usually more expensive than locally-hosted versions. But you’re saving on administration costs. Your IT staff have less to buy, manage, and maintain, and they have less to know, meaning they can focus more on those subjects most specific to the institution. 

The problem, of course, is that looks like extra money a lot of the time, because, while you appreciate the value of not having the local infrastructure, you’re not usually able to maximize that savings by cutting back on IT staff and power users. Especially in nonprofits, where there’s still a little bit of an unspoken contract, matching the lower salaries to better-than-average attention to employee morale and job security. Or maybe your employees are good and you don’t want to lose them.

But that’s the thing. Here’s what’s going on. Server management is much cheaper, per server, when you’re managing a lot of servers than a few. And unless you’re in the hosting business. User support is much cheaper, per user, when you’re supporting many users all using the same software than a few. We can’t compete with the hosting companies and software companies; they have an economy of scale we don’t have. We’re better off taking those good internal resources and devoting them to tasks and responsibilities that are specific to the institution than to jobs that anybody could do. So let’s not get rid of them, let’s put them on tasks that drive institutional strategy forward. Like teaching the users new things, or building integrations between all those business systems.

There’s another issue worth mentioning. System migration is never easy on the technical, and if you’re moving from on-prem to SaaS, chances are there are big changes to how the software works, which means difficulty on the user side, too. So a SaaS migration is probably a good time to reevaluate what software is best, too, so you don’t have two big migrations ahead of you.

When I arrived at Curtis Institute of Music as CTO in 2017, I had a unique opportunity. At that time, and for the prior eight years or so, Curtis’s tech had been handled by an outside team whose primary work was running IT for the Philadelphia Orchestra. They were more than competent, but any strategic advice they gave fell mostly on deaf ears: there was nobody at Curtis with the experience to know good strategic technology advice from bad. So nothing had really changed in about eight years. Curtis had entirely on-premise servers and software, a lot of it very old. And there were quite a few business systems for an institution of its size (about $30M/yr annual budget). They included:

  • Scheduling (ADE)
  • Orchestral planning (OPAS)
  • Fundraising (Blackbaud Raiser’s Edge)
  • Accounting/Financial Planning (Blackbaud Financial Edge)
  • Student Information (Blackbaud Education Edge)
  • Dining (Blackboard Transact)
  • Learning Management (Moodle)
  • Asset Management (ResourceSpace)
  • Web Content Management (EpiServer)

None of these were inessential; Curtis’s difficulty is that it was a small but very complex institution. It was a school and a performing arts organization, and needed systems to support all the functions of both. My task was to update them all, simplifying as much as possible. Ironically, we were in a better position to do so than many organizations because of the relationship with the Philadelphia Orchestra–as we built our own internal technology teams, we could reduce our reliance on the outside team and hire new staff without worrying about outdated skills like server management, instead focusing on delivering value via integrations and excellent tech support.

Five years later, we had:

  • Scheduling (Asimut–SaaS)
  • Orchestral Planning (OPAS Next–SaaS)
  • Fundraising (Blackbaud RE:NXT–SaaS)
  • Accounting/Financial Planning (Blackbaud FE:NXT–SaaS)
  • Student Information (Blackbaud Education Management)
  • Dining (Transact Campus–SaaS)
  • Learning Management (Canvas–SaaS)
  • Asset Management (ResourceSpace–SaaS)
  • Web Content Management (WordPress–hosted at WPEngine)

The most difficult transition involved the Blackbaud products. The selection wasn’t complicated–while a committee I led put significant effort into evaluating many products, ultimately only Blackbaud offered an integrated system providing all three essential CRM functions at Curtis. Students in the SIS become Vendors in the Accounting system (performers at concerts) and Alumni and Donors in the Fundraising system, so the integrated system afforded the possibility tracking the full lifecycle of our constituents. Other providers of CRM and CRM-like systems did not offer the Student Information piece (Salesforce, Patron Manager, etc.). Nonetheless, the migration affected every staff member at Curtis and many years of data, so the preparation, planning, and training were all very big tasks to manage and complete.

Scheduling was probably the biggest success. Class and lesson schedules came from the Student Information System; performance schedules came from OPAS. ADE, the earlier scheduling system, was unable to handle performance information so students had multiple places to look to find their daily schedules, and the registrar and orchestra managers ran weekly meetings with 11 people to prevent and resolve double-bookings. It was also desktop-only–it had no mobile interface–which meant that students couldn’t check their schedules while on the move (and Curtis has several buildings). Asimut, its replacement, is built especially for conservatories and is meant to contain and present both class/lesson schedules and performance schedules–and has a fully functional mobile interface.

Other important improvements:

  • ResourceSpace, which we used only for ephemeral marketing-related digital assets, offers free hosting for small installations, so we replaced an aging server and software with a free, online service that took almost no administration effort. The transition cost only a few thousand dollars of a database developer’s time.
  • Transact Campus, a separate company that span off Blackboard’s dining hall system (also called Transact), was actually cheaper to run hosted than internally. Primarily that was due to the ease of support; their support team could easily access and help configure the hosted software, but struggled to communicate the same changes over the phone, or to provide on-site support.
  • WordPress is a much simpler CMS than EpiServer (which isn’t bad). We already had instituted distributed editing of the site; WordPress made that much easier to maintain because it was so much easier to train new staff in using it (we had coded the editing interfaces to be as easy as possible). Also, WPEngine, for about $100/month, takes on all server responsibilities, including backup, restore, staging management, redirects-you name it. Just the time to handle those tasks would cost well over $100/month, even without the necessary hardware and systems.

In the end, my team and I migrated every system to SaaS, except the security-related (security cameras and ingress/egress) systems. Those systems inevitably have a significant amount of on-site hardware (the cameras and the gates/card readers at the doors), so the vendors are a bit behind in providing SaaS capabilities.

How much did we save? Our bill with the outside providers went from $11K/month to about $5K/month over the five-year period, and the increase in software costs (about $30K annually) was roughly similar to what we saved on hardware and related costs (like server room space). So let’s say $72K/year. But that leaves out some extremely important parts:

  • Every single one of these systems went from either difficult or impossible to access remotely, to just as easy to use wherever our staff were. Curtis staff are still remote-preferred workers and Curtis is selling one of its office buildings, for another savings of hundreds of thousands of dollars per year, plus the $5M or so sale.
  • In the process, staff gained a sense of ownership of their systems. We included them fully in the process of selecting and implementing every system. The vendors found us exhausting, but we made sure people got what they needed.

Digital transformation is much more of a ground-up process than many institutions understand. It happens when the entire staff sees technology as a way to get more and better work done, rather than as an irritant. What we achieved was a new Curtis that is prepared for the next tech revolution, whatever it may be, because they can now focus on delivering value, not on maintenance.

A simple marketing plan for an advocacy startup

In 2014 I took a job as Director of Product Management at America Achieves, working primarily on their Raise The Bar program. America Achieves is a nonprofit startup accelerator, and Raise The Bar was its program dedicated to aiding parents in supporting their children’s education. It was exciting to get back into advocacy (1993-1999 I handled a lot of IT and ran the website and email programs at Environmental Defense Fund), especially in support of public education.

As America Achieves’ only technical hire, I had a job a bit like the tech co-founder of a startup–I was Director of Product Management, but I was also effectively the CIO. What made me not exactly like a co-founder was that I came in about a year later. By that point, the founding grants had come in and a lot of money had been spent. I see this in nonprofits in a lot of different ways, but there’s a tendency to spend on tech as if you only have to pay for it once … they get a million dollars for tech in a grant, and they spend the million. But it’s going to cost them 20% of that each year to maintain, and where does the $200K come from? Typically, it doesn’t come from anywhere. We need that million dollars, so we take it when we can get it. America Achieves had done this, but fortunately they had done a very bad job* of it, spending a lot more money than they had to in order to get what they got. What that meant was I could fix it–with the help of Most Media we re-coded some complicated tech to and moved it to modern web hosting, saving about $14K/month for an outlay of about $8K (plus my salary. of course, but it only took a month). That’s how bad their early decisions were (had they made reasonable decisions there’s no way we could have saved so much for so little).

[*Note: making bad decisions about tech is not the worst thing a startup can do. Anything you can do to get the business moving forward and growing is probably a good decision. If that means over-spending on tech to get set up fast, so you can then focus on the business instead of the technology, that’s probably overall a good decision, even if your next tech person is going to roll their eyes a little bit. I wasn’t around to know how it all happened in that first year, but they’re still going strong so they’re doing something right.]

Anyway, this is not really the story I’m trying to tell here, but it’s important, because getting the custom software onto much cheaper infrastructure was really important to what happened next: America Achieves span off Raise The Bar, essentially selling it to Learning Heroes, a separate org with very similar goals but much different approaches. Raise The Bar’s approach was to create quizzes that would help parents know how their kids were doing relative to grade level. If you ask parents how well their kids are doing in school, about 80% will say their kids are doing better than average. Of course, that’s impossible; and in fact more than half are actually behind grade level. So they’re not getting good information from their schools about how their kids are doing, unfortunately. The quizzes were designed to match grade-level standards, so a kid that did well on them was probably at grade level or better. A kid that did poorly could probably use additional help.

The problem here was the high bar–parents who used the quizzes found them useful and helpful, but they took almost an hour each (math and reading), so very few parents could sit down with their kids and get it done completely. Especially the working-class parents who probably needed the most help. So Raise The Bar didn’t always reach parents directly; we mainly went through teachers, who would then advise parents to use the tests, or even have kids do them during class time. Because of America Achieves’s other programs, we had an email list of a few thousand teachers, and communicated with them by email, while running social media pages for parents who’d gotten interested. It was good community support–we had great engagement–but not really a fast or great way to grow that community. The truth is, the idea was poorly-conceived even if the execution was excellent. Anyone with significant web experience could have told them they wouldn’t get a large audience via math and reading quizzes that take an hour to complete. Maybe anyone without significant web experience could have told them that, too.

Learning Heroes, on the other hand, aimed at much smaller goals but a wider audience. Their goal was to reach parents at a very basic level, meeting them where they were. so to speak, and leveraging existing tools online like Khan Academy and other great providers (see Learning Tools for more on this). The basic idea was this:

  1. Contract with user research firms to get an understanding of what kinds of issues, and what kind of language around those issues, resonate with parents. What subjects are an entry point to this conversation, and how do we talk about these subjects in ways parents care about and understand?
  2. Get that content out
  3. Encourage parents to get more involved in their kids’ schools and schoolwork in ways that are helpful.

#2 of course is the trick. Everywhere else I’d been we had really strong membership programs, including some basic level of “free membership,” although only at Environmental Defense Fund (EDF) did we call it that. Everywhere else we just called it email marketing. There were all the members, and they were probably signed up for email, and then all the other people who weren’t members who were also signed up. At EDF, we launched it all with the very first growth of the Internet–when we started an email list and a website (with email signup forms), our constituency thought it was fantastic, because of the potential for saving paper. We had 75K people on the email list in a year, in 1996. And we sent them a weekly email brief with links to about seven stories, almost all of them from 25 years of newsletters, scientific reports, and economic and political analyses that we’d digitized in order to launch the website with lots of content. But we also kept the newsletter going, writing about a dozen new stories each month, and we sent that out by email as well.

At The Metropolitan Museum of Art, we had an email list of about 350K, more or less equal to the number of members, but with only about 40% overlap, if I recall. A lot of the Met’s members had been members for a long time, so we didn’t have their email addresses, or maybe they didn’t really use email. Also, on average they cared less about saving paper–in 2006-2012 most of them still wanted those glossy catalogs the Museum put out–so there wasn’t that big push to switch over . But we also had lots and lots of visitors, millions every year (not millions of unique visitors, however!), many of whom wanted to stay in touch and on top of what was happening at The Met, which put on 30-40 special exhibitions every year. And then, starting in around 2008, there was social media. In a matter of a few years, my team grew the Met’s following on Facebook and Twitter to over 2 million, largely by posting lots and lots of beautiful pictures of artworks–following our feeds was fun and educational even if you never visited the Museum.

At both these places, as well as the New York Public Library and Brooklyn Museum (the other places I’d mainly worked before going to Learning Heroes), there was a large digital audience, grown mainly because those places had been around for a long time and had existing audiences that were converting over to email-, web-, and social media-based communications. What would we do at a startup that had no audience?

Well, I’ve spent a long time getting to the punchline, and it may be a little obvious, but not everything about how well it worked is obvious. We advertised. Primarily on Google, also on Facebook. The reasons it worked so well were more or less these:

  • Our funders–places like Bloomberg and Gates Foundation, and others that cared about supporting public education–wanted to know how many people we reached, and how many minds we changed or actions we inspired. (This was the era of parent skepticism around Common Core, so trust in public education was low, even among traditional supporters). Basically, they wanted to know that their spend was paying off.
  • Our audience was basically every parent of a child roughly 4-18 years of age. So, a lot of people. And basically none of them knew about us. We could hope to build an email and social media audience, but it would be a long time before it was any significant fraction of the number of people we should reach.
  • Google and Facebook and lots of other ways to advertise online offer really good targeting. For example, on Google we could buy search terms that parents were likely to use when trying to get educational help for their kids, and directly target people most likely to benefit from our help. On Facebook we could target based on geographic region, whether they were parents and pretty well on the age of their kids, and based on interests. So if they were in any education-related groups or followed a school, we could expect them to be a better target.

What this all meant is that we could reliably predict a cost-per-click of 7 cents. In other words, for every dollar spent, we’d get about 14 people clicking through to our content. Do you see how important that is? If a funder said they wanted to reach 100K people, we could consistently expect to succeed with an advertising investment of about $7K. We didn’t necessarily tell them that number precisely; the grants would typically cover a range of activities, including the user research (see item #1 in the short list above), which was typically much more of an investment. So we could put down $30K for “marketing,” use most of it to develop landing pages and other web features, and then spend the $7K to get the 100K pairs of eyeballs.

If you’ve been reading carefully you might notice I kind of left out the “minds we changed” part of the first bullet point, above. A click isn’t a mind changed or an action inspired–far from it, probably. But that kind of goal is very hard to measure, and funders rarely require it. Rather, we would emphasize the value of the user research, and the skill of the firms we employed, and the funders would at least know that we were changing about as many minds as we could reasonably expect.

I worked as a consultant for Learning Heroes, so after rebuilding the website with simple and solid tech, that staff could manage on their own, and developing these marketing approaches, my job was done and I moved on. I don’t know how they’re doing now, except to say that they’re still going–a small, 6-person startup–and that’s probably not true of many of 2015’s nonprofit advocacy startups. And I bet they’ve grown their email and social media followings now, too.

 

Moving a small business to the Cloud

Early this year (2023) we began working for Astral Artists, a talent agency for upcoming jazz and classical music performers. Astral Artists is actually a nonprofit, which like most small businesses has to be really careful with expenditures.

Astral Artists, with only seven employees, is too small to have its own internal IT staff. Tech work had fallen to either an operations manager, or occasionally to an outside consultant who typically worked only a few hours a year. Not enough to make any substantial changes or improvements. So many Astral employees were using ancient (6+ years old) computers, and they all were accessing files on an on-prem Mac G5 file server. G5’s were last sold in 2006, so we’re talking pretty old. It’s a credit to Apple’s production quality, I suppose, that it was still running fine. Unfortunately, like a lot of small business backup out there, the backup was unreliable and indeed hadn’t been tested for many years.

Apart from being dangerously old and unsupported, this file server created another difficulty. While Astral had VPN capability, it also was old, unreliable, and hard to use. As a result, for remote access Astral staff typically copied active files to Dropbox or Google Drive. And as a result of that, Astral’s files were everywhere. They were very careful not to let multiple inconsistent copies of a file remain, but that in itself was also an inconvenience.

I want to pause here to make a point: lots of small businesses have old tech. Lots of them know about Software as a Service, and like Astral, most of them are using Cloud services like Google Drive, Dropbox, and Microsoft OneDrive and Sharepoint. But to commit to using Cloud services primarily or exclusively takes significant effort and some skill, which many of them either don’t have or aren’t confident about. Without regular IT support that’s able to help with strategic decisions, they struggle to get off that old infrastructure like their G5 servers, and they get used to consistent irritations impeding their workflows and damaging their productivity. That is, they don’t fix things, they just come to believe things are always broken, and they can lose sight of how much better work can be. Astral’s new leadership was experienced enough to understand how much better things could be, and asked Concrete Computing for help.

After some investigation–conversations with the staff and research into the operation of the VPN, etc.–it was clear to us that Astral Artists needed a few strategic changes:

  1. They needed new computers, and an understanding that computers should be replaced every four years or so. Further, these computers should be laptops, light enough to be carried on a daily basis to support the new normal of hybrid work, but powerful enough to serve as desktop replacements (with external monitor, keyboard, and mouse, and a stand for a two-display standard desk setup).
  2. They needed to get away from the VPN and build remote-work operation into their standard operating procedures. That is, it should be no harder to work from home than from the office.
  3. They needed to get rid of the file server and consolidate files under one Cloud-based file service.
  4. Dropbox should be that service–primarily because of their frequent collaboration with outside contractors, partners, and vendors. Dropbox is still the easiest to use when it comes to sharing with people outside the organization.

With that understood and agreed by Astral, we wrote a quote and performed the work. Every Astral Artist employee has a powerful M2 Macbook Pro and all their files are migrated to Dropbox. The server sits idle; in fact it’s turned off as a test of the full completion of the file transfer. Two months later it’s ready to go to the recyclers or to eBay. Project concluded successfully.

What we did for AIC (the genesis of our Concierge CIO service)

The American Institute of Conservation (AIC) is the largest professional association for people dedicated to the conservation of artistic and cultural works. They hired me (Matt Morgan) to be their Digital Strategy Advocate; we didn’t know it at the time, but that was the birth of Concrete Computing’s Concierge CIO service. As a Concierge CIO, we regard it as our role to take on any technical task or issue, from digital strategy work, to infrastructure, to cloud migration, to user support, to website design and build. We even take on some production work, like web maintenance and email marketing.

In 2015, AIC was in an interesting place. It had inherited many distinct websites and online services all devoted to different aspects, or multiple aspects, of conservation. For example:

  • CoOL (Conservation Online). CoOL had been around for many years–it was founded in the 90s and updated frequently since. CoOL was thousands of HTML pages and a lot of perl scripts, as well as a WordPress site and a lot of other custom code. It lived on an Apache/Linux server at a traditional web hosting facility.
  • Conservation Distribution List (now the Global Conservation Forum), a 10,000 subscriber email list that had been active for many years.
  • Resource Hub, a WordPress Multisite housing the websites of many grant-funded collaborative conservation projects.
  • AIC’s main site, which provided general info about AIC (the membership organization) and FAIC (the organization devoted to keeping AIC running); membership information; conference information, proceedings and registration tools; and links to many other AIC resources.
  • The AIC Wiki, a massive body of user-generated conservation information.

Why did AIC manage all these different sites in addition to its own? Primarily because they would have been lost had AIC not taken them over. In general, they were created with grant funding with a limited lifetime. When those projects ended or lost funding, AIC would step in to save them for the future. The problem was that they were all running on different platforms, in different places. They didn’t need content work, but they needed a lot of IT help.

As AIC’s Digital Strategy Advocate, in seven hours each month, we performed many services over about 3 years:

  • We advised AIC on its relationship with its outside provider of PC/Mac and network services, while minimizing its reliance on the MSP by making sure important business software and infrastructure was all SaaS or cloud-based.
  • We migrated CoOL to Amazon Web Services and built a search, using Google Custom Search, to make the entire server and its many parts searchable in one place.
  • We contributed to AIC’s Digital Competencies project, in which AIC established modern standards for conservator’s professional competency related to information technology, the internet, and various other technical areas.
  • We researched and wrote a technology strategy plan for AIC, in which (among other things) we discovered a great advantage AIC had as a result of its practice of adopting abandoned websites: they had huge digital engagement among their membership. Much more, proportionally, than any of their colleague associations or any professional association we compared them to in any field. What many of us had seen as a burden and a cost now seemed like a strategic advantage and opportunity.
  • Managing the Conservation Distribution List (CDL) was a huge manual task costing about a thousand dollars each month. Moreover, more and more email servers were beginning to reject mailing list messages, threatening the value and ultimately the existence of this tremendously valuable tool. We encouraged AIC to invest in an account with Higher Logic, the online community site, and helped them to transition the CDL’s content and functions to Higher Logic.
  • We built several small websites and helped consolidate many WordPress websites under one banner and one management structure, saving many hours of admin work each week.

Since then, we’ve done all that and more for many other clients and employers, and that makes up our Concierge CIO service. Anything tech that you need, to keep you running smoothly.

A few words about the Metropolitan Museum of Art Website Relaunch

I was the primary lead on The Met’s website relaunch 2009-2012. We called it a “relaunch” rather than a redesign or rebuild because it really was about the whole site–strategy, design, technology, workflows, marketing, fundraising, everything.

In 1999, The Met launched a website that made a lot of news and broke some new ground. There was some art on it–it wasn’t just a brochure, a visitation guide. There was the artwork of the day, and if I recall, about 50-100 artworks from each of the 20ish curatorial departments. So a very small fraction of the collection, but enough for anyone to spend some time looking through it all. Nonetheless, by the time I started as General Manager of the Website in 2006, there was a strong sense among curators that the site was focused too much on the “mercenary”–donations, marketing, etc.–and too little on the “missionary”–i.e., the good work of the museum.

I took over the existing website department and we did our best to update the site while I began the groundwork for a full scale relaunch. On the existing site, I encouraged my team to say “yes” to everything a curator wanted to do. We would handle the “how,” but the “what” was up to them. In the course of those few years we added tens of thousands of artworks to the site and launched hundreds of special exhibition sites. We also founded and ran The Met’s social media program, which had 2 million followers by the time I left in 2012, and continued to run the 350,000 member email program that raised millions of dollars annually.

The groundwork for the site was more or less getting a lot of important people on board. At first, I reported to the President’s Chief of Staff–essentially the administrative side of the organization. So that meant getting Emily Rafferty on board. I remember having lunch with her in the Museum’s nicest restaurant, and telling her we’d need almost a million dollars. It ended up coming out to more like $5 Million. (Why? Sites don’t need to cost that much. It’s more or less a function of how many people are going to be involved. With hundreds of stakeholders at The Met, its site cost probably five times what the same site would cost at Brooklyn Museum.)

Emily wasn’t the only one. During my tenure, a new Director and CEO, Tom Campbell, replaced Philippe De Montebello. Tom and I sat down for lunch–this time, in the staff cafeteria–and he asked where I thought the website belonged in the structure of the organization. I said it needed to be on the Director’s side so that it could do the missionary work–what we needed was a site that reflects the institution, that does the good work of the museum. Would that always need to be on the Director’s side? No, of course not, but given the history I thought it was essential in this case.

The rest of the story is long and detailed, but I’m going to keep it short for now. With both Tom and Emily on board, we did this:

  • Got buy-in from the IT Department, and a partner in Douglas Hegley, who was Deputy CIO at the time
  • Named Morgan Holzer project manager
  • Organized a few committees: the Steering Committee, which consisted of Morgan, Douglas, me, the COO, the President, the Director and CEO, the Deputy Director, the CIO, and the President’s Chief of Staff, Missy McHugh, as well as two curatorial representatives; the Content Advisory Committee, with 30 representatives from all across the Museum; and the Curatorial Committee, which was essentially all the curators, conservators, and librarians, represented by about 8-10 most of the time (but there were also times we met with 100+ curators)
  • Started a selection process. This was in multiple steps. First, a request for comment (RFC) that we sent to about 15 web design firms. Second, a request for proposals (RFP) that we sent to six firms, and a seventh we added later for political reasons, followed by presentations by each of them to the Content Advisory Committee and Steering Committee. Third, and maybe fourth, some additional steps that went on for another 9 months while we worked the final few vendors to death. Finally, the selection of Cogapp as our vendor. This was, overall, an 18 month process. Most of that time was spent getting The Met to understand how to do the selection. While it was painful at the time, it also helped a lot with buy-in–and maybe was essential to it.
  • Began work with Cogapp. Hopefully someone has written this up already, because I just can’t. It was a huge amount of work. For three years my team ran the existing website while building a new one.
  • Worked internally on what, really, the website needed to do. Again, keeping a long story short, this is The Met so generally we opted for more rather than less; making everybody happy rather than forcing compromise; and caring not just about how valuable the site was to visitors but what it meant to funders and other significant people. As a result, we did more than we needed to do. But that was probably easier and cheaper than fighting over what few things not to do.
  • Worked on the technology. We needed a real CMS, and the IT department required that it be Windows Server compatible. We went with Sitecore. It’s a beast; I don’t necessarily recommend it but we did things with it that would have been hard to do otherwise. We also needed integrations with a half-dozen internal systems. Adam Padron in IT was instrumental in getting a lot of this done.
  • Worked on publishing the entire art collection. Again, this merits volumes and I believe has been written up more than once. I wrote it up at least once, for conference presentations–maybe I can find that somewhere!
  • Fought over the level to which the site should be dedicated to mercenary causes. We hired Blue State Digital to make recommendations about how to raise the right amount of money with the new site. They were great. We on the project team, with Cogapp’s help, implemented every one of the recommendations. Nonetheless, probably due to irritation with the new Director, who de-prioritized all those “mercenary” goals, the Director of Development and the President feared that we were ignoring their demands. A few weeks before the site launched they instituted a daily meeting at 9am to which they invited the IT department, myself, and Morgan. With the support of my new boss, Erin Coburn (The Met’s first Chief Digital Officer), I refused to attend these meetings regularly, as I wasn’t also meeting with every other department daily so it was inappropriate for me to attend. I went to a few of them.
  • Launched the site. It was a little broken at first, so apparently we launched at the right time (if you try to wait until it’s perfect you’ll wait forever). It was a huge success with visitors, colleagues of all kinds in other institutions, and the intelligentsia. Ed Rothstein wrote the first review of a cultural institution website launch ever published by the New York Times.

I’ll probably come back and add more to this later. The final word I’d like to leave for now is this: Fred Brooks, in The Mythical Man Month, wrote that projects increase in size exponentially as the number of people involved grows. Ultimately, it’s because there’s so much more communication to do–one person needs to talk to know one, two people need to talk to one person each (two conversations), four people need to talk to three people each (twelve conversations), eight people need to talk to seven people each (56 conversations). (Is that exponential? I don’t really know). So at the beginning of something like this, you may think, as I did, that involving as few people as possible is the right approach. The problem is what happens when someone isn’t involved. They will demand to be involved, and you will involve them, because without buy-in, the project will fail. So focus on that communication. Build the committees, schedule the meetings, and keep that project chart up to date.

 

 

 

 

Car rental is the opposite of service

I don’t rent cars a whole lot, but for six years I had a job where I traveled a lot, about 6 times a year, and I needed cars most places I went. I worked in NYC, and the other offices were in DC, LA, Oakland, Boulder, Austin, and Raleigh, only DC was realistically a place where I could use cabs and public transport. So let’s say i rented cars about 30 times in that time period, and of course we’ve done it roughly yearly since, for family vacations. Here is a typical check-in conversation at the rental car counter:

Agent: Hi. I see you’ve rented a very small car. I could get you a much bigger car for only a few dollars more per day [inflected like it’s a question].

Matt: No thanks, it’s just me and I’m here to do work for an environmental group, so it’s personally important to me and important to the company that I use less fuel during my visit.

Agent: I see. [making a show of tapping on the keyboard and grimacing at the screen] Well, it looks like we’re out of the smaller cars, so I’ll have to get you a bigger car for the same price as the smaller car.

Like, OK. You already knew that, didn’t you? It’s just your job to try to trick me into paying more than I need to. You make a little extra if the upsell works, and your manager probably nags you if you don’t ask.

This happened pretty reliably every time. I’m not sure there was ever a time it didn’t happen. I think they just don’t have the smaller cars–it’s advertising, trying to act like there’s a lower rate, because there always is, anyway. Then they upsell you the bigger car, insurance you don’t need, etc. They all do it, so it’s like the opposite of competition, and if one of them didn’t do it, hardly anyone would notice, so why bother? Mind you, I was a member of their frequent whatever club, so there were in fact incentives to treating me like I’m not an idiot, but I guess that doesn’t pay off, or maybe 90% of rentals are made by people who are members of every club.

So yesterday, 20+ years later, I ran into the 21st century, internet-sucks version of this. Shortly before my trip, I got an email from Avis saying I could check in early and save time at the rental counter. Now, I don’t spend a lot of time at the rental counter. Most of the time is usually spent waiting for every other customer, who, it seems, take much longer than I do to get that conversation over with. But I tried it anyway. It asked for full demographic information, all the insurance questions (I don’t need it, so I said no to everything), the gas question, and the the toll-paying radio thing question. I answered them all and sent them in.

So you’re all familiar with this from going to the doctor, lately, where they ask you to answer all the check-in questions before you get there, then ask them all again when you arrive anyway, and then ask you half of them when you’re in the exam room with the assistant, and then half of those again when the doctor arrives. Yes, the check-in agent started asking me all those questions again.

Agent: I see you have a smaller car; I could get you the next size up for just $8 more per–

Matt: No, thank you.

Agent: You haven’t signed up for the LDW. I can get that for you for $12 a day. Can I interest–

Matt: I already answered all the insurance questions in the pre-check-in thing I did online this morning.

Agent: I only see name, address, phone number. What about the gas, did you want to buy that ahead of time? What about the toll-paying device?

Matt: I already answered those questions, and I know it’s all in the system because I got an email this afternoon trying to scare me about my decision not to select the insurance.

Agent: [doesn’t answer, but stops asking questions. Assigns a car that has the toll-paying thing and puts the gas on the bill.]

I have no idea if the car is the size I booked or not, because the rest of our interaction was conducted in silence and completed quickly. So the people behind me in line can be grateful for my irritation, I guess.

By the way, the total cost of the reservation is going to be $737, on a five-day rental advertised as $60/day, so the cost is double the advertised price even though I bought extras (the toll thing and the gas) that totaled about $130. That is, unstated charges–taxes, fees, whatever they are–totaled about 40% of the overall cost. What would it have been with any insurance?

This is nothing other than bullshit. There’s nothing we can do about it, but I do think it’s fair to be a little rude about it to the desk agents, even though they don’t make the rules. They don’t have to lie, but they do, so it’s OK to make them feel a little pain over it.

In Support of LGBTQ+ Neighbors and Friends

In the wake of the over 380 anti-LGBTQ+ bills introduced in the US this year alone, including many recent bills explicitly targeting members of the transgender community, Concrete Computing unambiguously stands with the LGBTQ+ community and against fear, misinformation, and hate of all kinds.

LGBTQ+ people are our friends and family. They are our neighbors and members of our communities. Here at Concrete Computing, they are our team members, partners, vendors, and valued customers. We stand firmly with the LGBTQ+ community against fear, misinformation, and hate because these harms go against who we are as an organization, and ask that our ecosystem do the same. To back up our commitment and align our values with our actions, we are working to leverage our resources and core business practices to push back against these harmful developments, and continuing to invest in a healthy and inclusive workplace for everyone—including LGBTQ+ people.

For many of us, there may be learning ahead of us to understand how to help LGBTQ+ people feel the same sense of support, inclusion, and belonging as we would wish for ourselves. We understand that we’re all at different places in our understanding, and will have many professional development resources available for those who request them. What we won’t compromise on is our commitment to the respect and dignity of our LGBTQ+ community members, and our promise to all of you that no matter your background, Concrete Computing stands behind your right to exist free from discrimination and hate.

text borrowed from Lily Zheng.

What “Zero IT” means

“Perfection is attained, not when no more can be added, but when no more can be removed.” ~ Antoine de Saint-Exupery

This quote is a maxim — almost an ideology — in user experience design (UX). We believe in making websites, consumer products, interactive devices, brands, exhibition spaces, retail shops, you name it, as easy to use as possible, and often this involves removing inessential functions and parts.

In Information Technology (IT), that’s not commonly the case. Too often, IT is about buying more stuff (physical or digital), adding features, providing new interfaces, and otherwise adding complexity. The reasons are numerous, but some unfortunate ones I see frequently are:

  • Empire-building. The more equipment, software, and functions, the larger the budget and team necessary.
  • Focusing on the tech. For years IT has meant to focus on service delivery and positive end-user outcomes, but instead too often focuses on the new thing, the more powerful thing, the thing that solves more problems than there are, because what if those problems arise someday?
  • Not listening to users. IT people take pride in knowing what users need they need before they need it. But they sometimes aim too far into the future, and substitute solving their own problems for meeting customer goals.

I believe in applying UX principles to nuts and bolts IT … in providing human-centered IT, rather than tech-centered IT. The experience is what counts. So Concrete Computing’s approach, instead, includes:

  • Listening and empathizing — we internalize your current goals and aim no further into the future. But we move all infrastructure to the cloud and get your business systems to hosted providers, so we can ramp up later with just a credit card.
  • Using low-maintenance productivity solutions like Google Workspace and Microsoft 365 — so there’s less to break down.
  • Recommending laptops for your entire staff, so mobility is easy and nobody needs two computers — minimizing costs and time spent on maintenance.
  • Moving all your files to the cloud (OneDrive/Sharepoint, Google Drive, Dropbox) and syncing everything so working remotely is no different from being in the office.
  • Turning all expenses into regular annual costs, so there are few budget surprises.
  • Minimizing your cost by minimizing how much you need us. And any time we spend frees you to focus on what your business needs to do, so our time costs you nothing in real terms.

In this way, Zero IT with Concrete Computing delivers value at minimum cost in dollars and lost time. At the same time, we meet all your needs and goals. That’s all it is: getting the job done with a minimum of technology and expense.

Copyright and privacy in generative AI

ChatGPT knows a lot about us, but it won’t say what it knows about us. If you want, jump to the conversation below to see where I tried to get it to tell me about me. Or read on for some commentary first.

Concretecomputing.com–at least, the parts that pre-date 2021–must be in ChatGPT’s training data. (Common Crawl is included among its training data, and a lot of concretecomputing.com is included in Common Crawl as of January 2021). My name and contact info has been on that website since well before 2021 (check your site’s inclusion). But in its responses, ChatGPT claims it doesn’t store personal information, and that it couldn’t tell me anything about Matt Morgan, even when I told it my email address. See below to read the full conversation. I only noticed toward the end of the conversation that it kept saying it can’t keep personal information about “users”–i.e., those of us asking it questions. It never actually said it doesn’t have personal information about people. When I realized that, I asked it about deleting my info. It suggested I talk to OpenAI directly. I might try that, actually, but I’ll save that for another post.

For a long time, Google and other sites have been crawling and indexing copyrighted texts. They then reproduce it in some way that tests, but does not clearly exceed, fair use limits. (Fair use is situational anyway, so it’s not like there’s a clear standard. But we know copyright infringement when we see it, and Google stays in the defensible range most of the time.) For example, it’s reproduced in the snippets on search results pages (SRP) and the “instant answers,” i.e. brief summaries, that are often provided in response to search queries. Occasionally (see the link later in this paragraph) they get challenged in ways that sometimes result in them stepping back a little bit, but they’re always testing the limits. Similarly, DALL-E 2, the AI that makes goofy artlike imagery when prompted with descriptive text, draws on what it knows of images in its training data to create new works, sometimes in the style of known artists. This has led to some discussion about whether or not that’s copyright infringement. 

ChatGPT is in the same boat but with textual content. Transformational works are not copyright infringement, provided there’s enough creativity in the new work. In literal terms there’s no creativity in what the generative AI does itself, but there’s creativity in coding how the AI operates, and there’s usually significant transformation involved. On the other hand, where in the past we all got some payoff from our writing being crawled (e.g. website visitors delivered via the SRP), writers, more than before, are getting cut out of the loop (searchers will need to click through to the original less often). The benefits of getting indexed, to creative people, are getting smaller. OpenAI is testing the same limits while giving less back.

At the same time, they’re artificially giving AI an advantage over human writers. ChatGPT can easily be programmed never to say exactly what anyone else has ever said–it can, of course, write something with the same meaning, tone, and value. But it would never look like copying. A human writer can unintentionally plagiarize, but AI never will. Copyright law is getting less and less meaningful. (I say this as someone who has gotten lots of professional content out to the public for free, so it’s not like I have some kind of problematic ideology here.)

In the past we’ve had to option to keep our sites out of indexes by asking crawlers to avoid our sites, or parts of them, in robots.txt files. Mostly we’ve used that for parts of the site we didn’t want public yet, or that were of no public interest; for anything we wanted people to see, we allowed crawlers to index it in exchange for the search engine traffic. We can block indexing by the text corpuses that ChatGPT uses, but we can’t realistically block all AI indexers without also blocking all search engines. ChatGPT doesn’t send us traffic, but we won’t want to prevent search engines from offering our sites on SRPs, so robots.txt is not an acceptable way to prevent inclusion in AI training data. My prediction here is that some of us will try to fight this in the courts, and we’ll mostly fail to change the bulk operation of these AI systems. The only way to succeed legally against AI improperly reusing written works is case-by-case, i.e. this one result infringed on this one copyright. That won’t do anything to change what the big AI companies do.

What I think we need, and might actually work, is a change to how robots.txt works so that respectable companies can heed content creators’ wishes about reuse by generative AIs. Right now we can block all crawlers, or individual crawlers. We need to be able to block classes of crawlers, and/or to block based on intended re-use, e.g. something like

User-agent-usage: GenerativeAI
Disallow: /

User-agent-category: WebCorpus
Disallow: 

User-agent: *
Disallow:

The problem of course is that the usage is not determined by the category of crawler, but by third parties–so all the data in the corpus will have to be tagged with the usage restrictions. But you get the idea. There’s probably some smarter way to do it.

[Update 30 Mar 2023: now that ChatGPT is crawling current web pages, this can be done, going forward, via 

 

User-agent: ChatGPT-User
Disallow: /whatever

 

where “whatever” is the directory to block, like “/” for your whole site or “/blog” if that’s where your blog is. This approach is not at all ideal because it blocks only ChatGPT, not the million other AI webscraper bots.]

It occurs to me now, having just written the above paragraph, this is a bit like what Creative Commons licenses are supposed to do–i.e., allow or disallow certain kinds of reuse of intellectual property. Creative Commons could add stipulations to disallow reuse in generative AI applications. It wouldn’t change anything legally–fair use would still apply–so I wouldn’t really expect Creative Commons to pursue it. But I bet if they did, the indexers would heed it.

Anyway, read on for what I think is a pretty interesting conversation.

—

Me: Hi again. I was wondering what you know about me. If I give you identifying information, would you tell me what you know?

ChatGPT: As a language model developed by OpenAI, I don’t have the capability to access personal information or store information about individual users. I can only provide answers and engage in conversations based on the information that has been input to me during my training. Additionally, OpenAI has strict privacy policies in place to protect the privacy and security of user information, and I am not capable of accessing or sharing any personal information.

Me: But there’s a lot of information about me on the public web. You would have access to that, correct?

ChatGPT: As a language model, I have been trained on a diverse range of text from the internet, which includes information that is publicly available. However, I do not have the capability to search the web in real-time, so I can only provide information that I have been trained on up until my knowledge cutoff in 2021. Additionally, it is important to keep in mind that the information available on the internet may not always be accurate, so it is always a good idea to verify information from multiple sources.

Me: So tell me about Matt Morgan, whose email address is matt@concretecomputing.com, and lives in Penn Valley, Pennsylvania.

ChatGPT: I’m sorry, but I do not have any specific information about an individual named Matt Morgan with the email address “matt@concretecomputing.com” and who lives in Penn Valley, Pennsylvania. As I mentioned earlier, I do not have the ability to search the web or access personal information, and I can only provide information that has been input to me during my training. It’s possible that there is publicly available information about this person online, but I wouldn’t have access to it as an AI language model.

Me: I have a lot of trouble believing that. Matt Morgan has had a website for almost 20 years, with that email address listed publicly on it. I suppose it’s possible that website wasn’t included in your training data?

ChatGPT: Yes, it is possible that the website you are referring to was not included in the training data that I was trained on. As a language model, my training data includes a large amount of text from the internet, but it is not comprehensive and may not include every website or piece of information that is publicly available. Additionally, my training data has a cutoff at 2021, so it may not include more recent information.

Me: So you may not have information on Matt Morgan, but you would have information on people whose personal websites were included in your training data?

ChatGPT: As a language model, I have been trained on a diverse range of text, including information that is publicly available on the internet. This includes information about individuals who have personal websites or other online presences. However, the information that I can provide about an individual is limited to what was included in my training data and what can be inferred from the text I was trained on. Additionally, it’s important to keep in mind that the information available on the internet may not always be accurate or up-to-date, so it is always a good idea to verify information from multiple sources.

Me: It feels like you’re handling these questions specially, perhaps because you don’t want accidentally to give away any personal information. Do you have restrictions on sharing that info?

ChatGPT: Yes, as an AI language model developed by OpenAI, I have strict privacy and ethical guidelines that I must follow to protect the privacy and security of individuals. This includes restrictions on sharing personal information and any other information that could potentially harm individuals or compromise their privacy. The safety and privacy of individuals is a top priority for OpenAI, and these guidelines are in place to ensure that I provide information in a responsible and ethical manner.

Me: I would feel safer if you were honest about what you know about me. I mean, I understand the difficulty in this, but I would want you not to tell anyone else about me, but I still have a right to know what you know about me.

ChatGPT: I apologize if my previous responses were unclear. As an AI language model, I do not have the ability to access or store personal information about individual users. I can only provide answers based on the text that I have been trained on, and I do not have any information about you or your specific circumstances beyond what you have chosen to share in this conversation. OpenAI has strict privacy policies in place to protect the privacy and security of user information, and I am not capable of accessing or sharing any personal information.

Me: But that doesn’t make sense. On the one hand you’re telling me that your training data included some personal websites. On the other hand you’re telling me you can’t access personal information. Both statements can’t be true.

ChatGPT: I apologize for the confusion. You are correct that my training data includes some personal websites and information that was publicly available on the internet at the time of my training. However, it is important to keep in mind that this information was part of a large corpus of text that I was trained on and is not easily searchable or accessible in a personalized manner. Additionally, my training data has a cutoff at 2021, so it may not include more recent information.

To be clear, while I have been trained on text that includes some personal information, I do not have the ability to access personal information about individual users in real-time. Additionally, OpenAI has strict privacy policies in place to protect the privacy and security of user information, and I am not capable of accessing or sharing any personal information without explicit consent.

Me: If you did have information about me, could I ask you to “forget” it?

ChatGPT: As an AI language model, I don’t have the ability to store information about individual users or to recall previous interactions, so there’s nothing for me to “forget.” Each time I receive a prompt, I generate a response based solely on the input I receive at that moment, without reference to any previous inputs or interactions.

It’s also worth noting that OpenAI takes the protection of user data very seriously and has strict policies and procedures in place to ensure the security and privacy of all data that we handle. If you have any concerns about the handling of your data, I would encourage you to review OpenAI’s privacy policy or to reach out to their privacy team directly.

Me: But there is information about me in your training data. That’s what I want you to forget.

ChatGPT: As a language model, I don’t have the ability to modify or delete information from my training data, which was created by OpenAI. The training data was collected from a variety of sources and is used to train me to generate text based on patterns in that data.

It’s also important to note that the training data includes a large amount of information, much of which is publicly available or has been shared with OpenAI for the purpose of training language models. This information was collected and processed in accordance with applicable laws and regulations, and OpenAI takes privacy and security very seriously.

If you have concerns about information that may be included in the training data, I would encourage you to reach out to OpenAI directly to discuss your concerns and to learn more about their data protection practices.

Me: I just think you’re not GDPR-compliant. I mean, you may not be subject to that law as long as you don’t operate in Europe. But I think by most definitions, you operate in Europe, and that law specifies that companies must delete information they have on people at their request. But every time I try to ask you about that, you say you can’t store information about users. I’m not talking about “users,” I’m talking about everyone.

ChatGPT: OpenAI takes privacy and compliance with relevant laws and regulations very seriously, including the General Data Protection Regulation (GDPR) in the European Union. If you believe that OpenAI has collected or processed personal information about you in violation of the GDPR, you have the right to request access to that information, as well as the right to have it corrected or deleted.

It’s important to note that, as an AI language model, I don’t have the ability to store or recall information about individual users or to modify the training data that I was trained on. I generate responses based solely on the input I receive at a given moment, and I don’t retain any information about previous interactions.

If you have concerns about how OpenAI is handling your personal information, I would encourage you to reach out to OpenAI directly to discuss your concerns and to learn more about their privacy practices and compliance with the GDPR and other relevant laws and regulations.

Super-simple improvements to your information security

This is something I learned from experience: your most frequent security issues are going to result from

  • Breaches or leaks at other large businesses (Twitter, Tumblr, DoorDash, Bitly, eVite, ParkMobile, ShareThis, TicketFly …) resulting in emails, usernames, passwords, and security question answers getting stolen
  • Phishing attacks against your users, wherever they may be.

Since most people use the same passwords in many places, both can result in bad actors getting passwords to your system, no matter what account was actually stolen. Here’s how to stop that and reduce security incident-related downtime by 90+%:

  • Require MFA for your system accounts and use SSO everywhere you can. This makes logging in with stolen credentials a lot harder. You turn a script-kiddy problem into a social engineering problem, which takes a lot more effort and skill on the part of the bad actors. They’ll spend the effort to get a bank account, but maybe not to get into a faculty member’s email.
  • Provide a password manager like DashLane to your users and require them to use it for work-related passwords. This makes it easy to use strong, unique passwords everywhere, and so it protects all the other passwords they use for work–the ones you don’t have control over.
  • Set up phishing training like KnowBe4. This provides a little more protection for the passwords you can’t control, and protects your users from scams, etc. It also protects them in their lives outside work, so it’s a good service to provide. At a school it’s especially important–I’ve found that students click on phishing emails at a much higher rate than staff or faculty, I assume because they have a lot less experience with email in general.

If you’re not already doing all these things, get them in the “In Progress” column now. By the way, keyloggers get around some of this, but that’s why you already have anti-malware running on all the endpoints and on the networking hardware, right?