a panel discussion with:
- David Martinsen of the American Chemical Society (ACS)
- Andrea Powell of CABI
- Howard Ratner of CHORUS – the Clearinghouse for the Open Research of the United States
- Heather Ruland Staines of SIPX
On April 29 in Washington DC at the STM Innovations Seminar, the STM Future Lab Committee launched its 2014 edition of the annual report on STM Tech Trends. This yearly exercise highlights the most prominent technology trends that committee members believe will have significant impact on scholarly and scientific publishing in the coming three years. A panel of technology directors and business development senior staff from STM member companies and organizations explored these themes with moderator Christopher Kenneally of Copyright Clearance Center.
For podcast release Monday, May 5, 2014
CHRIS KENNEALLY: Good afternoon everyone. My name is Chris Kenneally. I’m with Copyright Clearance Center and looking forward to moderating a discussion on tech trends for 2014 for the next 45 minutes or so.
A quarter century ago, a British computer scientist working in Geneva presented his boss with a proposal for a new kind of network able to link documents and data across the Internet. The manager had been gently encouraging his employer in this work, but he wasn’t ready yet to commit resources. So he wrote across the proposal, vague, but exciting. And Tim Berners-Lee would have to continue his work on his own.
Vague but exciting – thinking about the future will give you that kind of a feeling. It’s all very exciting. Google Glass, smart watches, 3D printing, and it’s also a little vague. The Internet of things. Which things? Is it my bathroom scale telling my refrigerator not to let me have the pudding tonight?
Today the International Association of STM Publishers has launched its annual edition of a report on STM tech trends. This yearly exercise highlights the most prominent technology trends that we’ve been hearing about that the STM Future Lab committee believes has significant impact on our industry in the coming years. And for 2014, we heard briefly about these three main trends. I’ll just tell you a bit more about them.
The machine is the new reader – it’s also the new baseball fan, as it turns out – in which we confront the impact from machine readability of information, the growing importance of metadata, and the potential for text and data mining, and creation of minable archives.
In 2014, data is big. And big data is even bigger. Ultimately the machine may prove, not only to be the new reader, but also the new researcher and author, analyzing academic publications and generating hypotheses on its own. One day it seems the machine may do it all.
The second theme is the return to the author in which we focus on the increasing role, at least for now, of the red-blooded human author, notably in a publishing environment where open access rules. Increasingly, the scholarly publishing ecosystem appears more like a scholarly author ego system in which demand is rising for improved metrics and greater transparency throughout the information workflow.
Finally the third theme for the day is new players changing the game, in which the Future Lab committee has caught in its net a menagerie’s worth of emerging companies and services available on the Web and other online platforms, including social media, authoring tools, open metrics, and data sharing. Many of these are innovative start-ups seeking to leverage open access information and to offer new discoverability tools to authors and researchers. While their names may be unfamiliar, you will certainly recognize others: Google, Amazon, Facebook, and Apple.
Now making predictions about the future is notoriously difficult, especially predictions about the future. I believe that was Niels Bohr who said that, not Woody Allen, though it sounds like a comment he would have made. And to help me with those notoriously difficult predictions today, I want to welcome my panel.
Moving from my right, Howard Ratner. Howard, welcome. Howard Ratner is executive director of CHOR, Inc. and leads its first service, CORUS, which is an acronym for the Clearinghouse for the Open Research of the United States. CORUS is a not for profit public/private partnership including publishers, resource partners, associations, and others whose mission is to increase public access to peer reviewed publications that report on federally funded research. It’s important to this discussion that Howard currently serves as the president-elect for the Society for Scholarly Publishing and was recently chief technology officer for Nature Publishing Group. He co-founded and chaired the not-for-profit Open Research and Contributor ID System we know as ORCHID. And he continues to serve on its board of directors.
To Howard’s right is Andrea Powell. Andrea, welcome. Welcome to Washington and the United States. Andrea Powell has been with CABI in the United Kingdom since 1991, initially in a marketing role then as product development director, and since 2005, as executive director of publishing. CABI is an inter-governmental, not-for-profit organization set up by U.N treaty. Its mission is to improve people’s lives worldwide by providing information and applying scientific expertise to solve problems in agriculture and the environment.
To Andrea’s right, we’ve already heard from David Martinsen, but we’ll tell you a bit more about him. David Martinsen has been at the American Chemical Society – and David, welcome – for 25 years working in various capacities in the publication division and in its IT department. And ACS, as you know, is the world’s largest scientific society. And in his current role, David is responsible for tracking new technologies and planning their incorporation into the scholarly publishing environment. He’s currently chair of the committee for printed and electronic publications of the International Union of Pure and Applied Chemistry.
And then finally on the far end, we have Heather Ruland Staines. Heather, welcome. Heather is vice president publisher development at SIPX, formerly the Stanford Intellectual Property Exchange. It’s a Web-based service to manage copyrights and deliver digital documents for the higher education marketplace. In her role there, she explores much about the future, including the nexus of academic publishing, library technology, and the future of e-learning. She is, among other things, on the board of directors for SSP. And she’s also a Google Glass explorer.
So before we continue with the discussion, we’re going to have everyone, just if you would, please, pose for a picture. So on three, would you say STM?
STAINES: OK, Glass, record a video. Everybody can wave. (inaudible) do it by the way.
KENNEALLY: We’ve got a video of that. Everybody, wave. Exactly. We’ll have that posted soon enough.
STAINES: There we go. (overlapping dialogue; inaudible) here.
KENNEALLY: With all of that, I do feel it’s important to give everybody a sense of everyone’s background on this discussion. And we will move to the first piece of it, which is – and we’ll try to get through these slides one by one, and then have some time for your questions – is the machine is the new reader.
And Heather, I’d like to start with you, because I know in your work, your concern is metadata. And that seems to be critical to this notion of the machine is the new reader. Tell us, what are some of the problems and what is some of the promise here? We’ll speak later about the role of new players and Google, not so new anymore. But Google itself plays a role, not only in your Google Glass, but in metadata, in fact a negative role. Tell us more about that.
STAINES: Well, I don’t know if I want to go so far on record as to say that Google plays a negative role. And there are folks here who are much better acquainted with metadata than I am. But one of the things that I do at SIPX and one of the things that I was responsible for at Springer, working on SpringerLink was making sure that we had good metadata.
And frequently what I hear from publishers is, in terms of corrections in metadata, that metadata goes out in so many different directions, and then it’s out of our control, that there’s a limit to what we can do to update it. Because at a certain point, it’s out of your control. And I know many times Google would actually overwrite publisher corrections to metadata. So that was the context –
KENNEALLY: OK, well, fair enough.
STAINES: – that Chris and I were talking about. But I think metadata is, in many ways, as valuable as the content. And there’s so many new players in the space trying to make use of metadata, particularly in terms of identifying open access from metadata. And right now I know that’s something that NISO is really in the thick of. But I think that’s crucial for all of us in our daily lives.
KENNEALLY: And Howard Ratner, for you, the notion of data, linked data, metadata, big data, all of the data types is again about the questions that publishers have with regard to their role in this play.
HOWARD RATNER: Right. So one of the things that Heather said really resonates, is, there’s so much information that’s out there. And as we were hearing just about statistics in baseball, there’s so much information about baseball. Right? So how do you get through that? How do you figure it out? How do you start analyzing that information?
And I think the key for me is making sure that that metadata is all transparent and accessible. And then once you have that together, then you can start doing some interesting analysis, and then start to make some decisions on it.
So I think for us as publishers, and for us that are in the publishing industry, we have to decide what levels of metadata do we want to play in? What do we want to make transparent? And what do we want to truly make open? I think it’s an interesting space that we’re in now and it will only grow.
KENNEALLY: Right. And how far along would you say that discussion is then with regard to the level of transparency?
RATNER: Well actually in my new role at CORUS, we’re starting to see new levels of transparency that we’ve never seen before. As an example, you know, what is actually hidden in those dark archives? What is the licensing of relationships that you might have and for reuse? These are all things that have never been really highlighted before. And I think these are important things.
And then you start talking about the data. And where is this data? What kind of data in it? Who created the data? What’s the provenance of the data? This is all huge. So I would say we’re at the very, very beginning stages of all this, especially when we talk about data.
KENNEALLY: Andrea Powell, at CABI, it’s important to point out that you’re not a primary publisher, per se, and that really does make a difference because up to a point, your readers at CABI have been organizing journal content and adding value that way. What will it mean that the machine is now a reader as well for you?
ANDREA POWELL: Right. Thanks for that. That’s absolutely right. And I think the machine becoming the new reader has particular resonance for organizations like CABI, where in our role as secondary publishers, our role is to provide that normalization of content such that it becomes discoverable, so that we can lead our users to full text content wherever it resides.
And increasingly, if much of the content that’s out there isn’t being read by a human eye, the importance of consistency and real true discoverability becomes critical, whereas in the past, you know, slight nuances between the metadata that’s applied from one journal to another might have still served a need. Increasingly, that’s not going to be the case. And we have to work harder and harder to apply that level of normalization across the vast corpus of material that we’re processing in our database.
And we have a taxonomy, a thesaurus, a controlled vocabulary in agriculture. But there are three controlled vocabularies in agriculture around the world. And so we’re working with partners in those other two organizations, the FAO and the NAL here in the U.S., to try and map those vocabularies together so that we can identify a core set of terminology that we can then make available in an open space, and therefore organize that material much more effectively.
KENNEALLY: Right. So in a sense, CABI is the guide through all of this literature. And in the past, the publications were sold to libraries. Today, you really are concerned with the user. And again, the work that you’re doing is thinking about a new type of audience using the machine data to help you develop products and to serve that new audience.
POWELL: Yes, that’s right. And it’s very exciting for us because we are able now, through better analysis of raw data to develop new services that genuinely help to improve people’s lives around the world.
And I’ve, previously at this meeting, spoken about our Plantwise initiative, which looks at the distribution of plant pests and diseases globally, and plots that information. And as we gather more and more and more knowledge about the presence or absence of plant pests, we can help the world to grow better crops, to lose less of what is grown to pests and disease, and really create new value from data.
And what’s fascinating and interesting for us as information professionals is the application of those publishing, or if you like, information management skills in these very, very practical scenarios.
KENNEALLY: Right. David Martinsen at ACS, you also are concerned about this flip from one type of customer to another. And you have to ask the question and answer, who are we trying to satisfy? What are we trying to give them?
DAVID MARTINSEN: Well, yes. I think the aspect of this little slide that intrigued me the most was about the cognitive computing and what it means for the machine to read and actually understand what the article is talking about.
And we were influenced a little bit in our discussion back in December by Frank Stein who was on the Watson development team. And he talked about some of the approaches they’ve taken in programming Watson, both in the artificial intelligence as well as machine learning, and incorporating both human and algorithmic inputs to create something that mimics human understanding.
I don’t know if it’s just another term for what was artificial intelligence 20 years ago or if there really is a new paradigm here. But if you look at what Watson was able to do on Jeopardy, the TV show, beat all the challengers, the top Jeopardy players of the past, there was some remarkable things there.
And Watson technology now going to school to become medical – well, not a medical doctor but perhaps a medical advisor, to assist doctors in diagnosing diseases and prescribing treatments.
And so there are some remarkable things that are beginning to appear. And I guess the question is, how will those emerge? Will they become an important factor?
One of the things that sort of seems very nice to me is that IBM talks about processing unstructured documents, and so all these things about tagging and metadata and so on, they appear not to need as much, which lightens the burden for many of us. But that is probably a step way out in the future.
KENNEALLY: What’s interesting about that, you used a phrase about mimicking human intelligence. And human intelligence certainly can use organization. It’s a helpful thing to have. But it doesn’t require a genius. It doesn’t require things being presented in a very neat and packaged way.
And so there’s a concern here, and I believe you have it, for, you know, carving out a space for serendipity even if the machine is now very much driving this process.
MARTINSEN: And I think one of the things that maybe we might touch on a little later, but reproducibility is a theme that has come up recently. And as you begin to – as you look at human processing of the same data for instance, humans may look at the same data and come up with different interpretations, different understandings.
If you had two different machines looking at this that started out at the same base point and then processed a whole bunch of information, got input from the same experts, but they actually reached the same conclusions as each other, or would we have maybe a non-reproducible algorithms or non-reproducible learning system amongst these cognitive computers? And would that be OK?
KENNEALLY: Answer the question. Would it be OK?
MARTINSEN: Well, I don’t know.
KENNEALLY: Heather Staines, for you, linked data is important. And if we think about linked data, I think it relates to the previous presentation with regard to identifying the need for new journals and in visual ways. Speak to that a bit more.
STAINES: So I got involved a couple of years ago through ALA with linked data. One of my committees, we wanted to do a kind of linked data workshop. Everybody thinks you need to have an enormous corpus of data before you can do anything with linked data.
So we got some volunteers, or victims as I would call them, from some libraries, one from Texas A&M and another from, I think from Brigham Young. And what they did was to take a small segment of information, and actually using a free tool called Viewshare, try to learn something new.
So as a result of, you know, working through different exercises – and we actually did a follow-up at the subsequent ALA – we learned about what’s being done with linked data, but also where there’s still work that needs to be done. Some of that might be identifying need for a new journal or a focus on specific type of articles.
But I also think in terms of, you know, using things like geo-location data and connecting that, libraries, while they may be able to put their catalogues online, one thing that you will not find – if you go in and you search for a book, you’re going to find it on Amazon. You’re going to find maybe reviews of that book. You’re not going to find if your public library has that book, or if your bookstore, you know, that’s actually half a mile away from you, has it.
So there’s a lot of connection we can do with both geo-location data and also publicly available and surface information in that regard.
KENNEALLY: Right. I’m going to move on to our second theme. You know, these slides, I was telling Eefka Smit to remind me of those days during the flower generation and the hippie bus that it was plastered with all sorts of bumper stickers for every concert and every concern they had. There’s a lot to cover in all those. We won’t get to all of it.
But here’s a big one, a big one certainly for all of us at Copyright Clearance Center, and that is the open access trend and sort of under the banner here, under the bumper sticker of the return to the author and how it – clearly important to you and your work at CORUS. But a point that you made in our preparatory discussion was that, what this is all about is author services before publishing. Talk about that.
RATNER: Right. So before publishing meaning that things along the lines of helping authors with metadata, helping them improve their services, those are all important things.
But the thing that I really get out of this one is, in the world of gold OA as we all have come to know it, it’s really about how the author is voting with their dollar. OK? In turn, by the way, that also means how the funding agency is influencing the author.
But this is really the important thing. This is a change. Because before, it was the reader that was influencing us. It was the reader that was voting with their dollar. Right? But now it’s actually the author.
And that’s a big change in this industry. And I think it could have a very big impact. And it is already. Already starting to see that it has a big impact with mega journals.
KENNEALLY: Right. And it’s having an impact on the publisher as well because the focus in the past had been on this deliverable, this product. What should the focus or tell us about the direction of focus now for publishers.
RATNER: Well, so not being a publisher anymore myself, I will say that, you know, providing the services and making that user experience when the author arrives on your doorstep as clean and as simple as possible is important. But at the same time, collect as much information as you can, because honestly, you’re probably only going to get one shot at it. So if you can get that information upfront from the author, but make it in a UI that’s pleasant and easy for them to do, they’re more likely going to come back.
KENNEALLY: Indeed. And Heather Staines, your own background is in the humanities, I believe. You’re an historian and you have a husband who is an English professor. And so for that particular piece of the academic publishing environment, open access is not as important, but it’s a discussion that’s not being had even there as well.
STAINES: Yeah, I mean, my friends from graduate school who are historians I think when, you know, SAGE, you know, bravely strode forward with SAGE Open and started to reach out to social science and humanities, you know, asking for publication, my friends from grad school kind of thought it was a little bit, you know, maybe of a joke. Was this some kind of a scam? That not only were they being asked to pay money to publish their article, but in fact if they got it in before the end of the year, it would be half price? And that their work would somehow be discounted was a little bit, you know, offensive to them.
But in the social sciences, more towards the humanities, there’s not a widespread familiarity with open access that I’ve found, unless you are at a school that has already mandated open access publication.
And my husband, who’s an English professor. He does 16th and 17th Century. He doesn’t have funders, so to speak. So if he decided he wanted to publish in an open access journal or if he wanted to explore some of the new ways for open access books, that would be pretty costly venture for him.
KENNEALLY: Right. Indeed. And David Martinsen at ACS, for some time now, you have published so-called hybrid journals where there’s an open access option. And you’re developing pure play open access as well. So you really are trying to serve people from across the spectrum.
MARTINSEN: Yeah, that’s right. And, in fact, our hybrid open access model has been expanded quite a bit in this year, that there are a number of different options available with both immediate gold OA, 12 month embargoed OA, and then additional add-ons for CCBY and CCBY NCND. And then (overlapping dialogue; inaudible).
KENNEALLY: A lot of acronyms there. We’re going to assume people are familiar with most of those.
MARTINSEN: Look up Creative Commons. And then we’re very much focused on the version of record as being the version that we really want most people to see. And so we’re also offering incentives to encourage authors to make that APC to get their article into the open. But by doing that, they will then get credits for future open access payments down the road.
So there’s a number of programs to help encourage that gold OA version, the version of record, as being the one that’s accessible.
KENNEALLY: Right. And David, what about this notion, it’s a catchy phrase of the author ego system. How is ACS prepared to serve that author ego?
MARTINSEN: I don’t know if I can speak authoritatively on that as to what is actually available or not. Sarah Tegen is going to be talking later on and will probably touch on that in her talk.
But I think there is very much a renewed focus on what are the tools that we can provide. What are the ways that we can get involved with the author earlier in the process, so at the time that it actually – to the point to where they’re writing and submitting their article, they’ve already had a relationship that has been smoothed by some of the tools and features that we offer them.
So in the old days, it was all about, what do we have on the website? How do we engage the reader? How do we provide the information in the best way possible? Now we’re much more focused in addition on what we can do to engage the author at a point that’s appropriate.
KENNEALLY: Engage them in such a way as to help them develop their own brand. And I wonder how it would rather – how that feels, that notion of the author brand in an environment where, up to this point, the publisher has been the brand.
RATNER: Well, in my opinion, it’s not so much about an author brand. It’s also the authors want credit for what they do. So in the work that we started a few years ago now with ORCHID, you really start to see, well, how do you disambiguate these folks? Because they want credit.
And they don’t want just credit with their article. The Future Labs group was talking about this before. They need to actually credit for a whole plethora of things. Because we were listening to some of our university colleagues and now the committees that decide whether or not these people get tenure or advancement or so forth are actually just now starting to look at things beyond the article.
So how do you identify when they’ve helped out with a data set? How do you identify when they’ve done some peer review? How do you identify when they’re published? How do you identify when they’ve written a blog or a tweet or what have you?
All of these things are now part of their scholarly record. And that is creating what is their real profile online and the scientific record. So it’s a real change.
KENNEALLY: So a sense of micro-attribution there.
RATNER: Mm-hmm, a sense of micro-attribution.
KENNEALLY: Even down to the tweets.
RATNER: Possibly. I had some question about that, but possibly.
KENNEALLY: I would have some question about that.
Well, let me move on to our third theme which really sort of catches everything else in its net, which is new players changing the game. And since you have already sort of begun the demonstration here, Heather Staines, I want to bring you back in, because Google Glass, and the whole notion of wearable, movable technology is a place that we can expect to see more development. But how quickly do you expect that to really become current?
STAINES: Oh, that’s a tough question. I’ve been thinking a lot, probably in the past six months, about why publishers should even care –
KENNEALLY: Right.
STAINES: – you know, about wearable technology.
I do think that, you know, given our focus on researchers that people will be creating content using Google Glass and other types of things, the GoPro and data that’s collected, you know, from some of the fitness devices. But also there is going to be consumption of content on these devices.
I have an app for my Glass called Field Trip which operates by geo-location. And if you are in a spot and there is information that’s maybe germane to what you’re doing there, you hear a chime and you have a chance to learn more. So if there’s architectural information about a building you’re in front of, walking around Charleston, you can see pictures of what that street looked like after the devastation of the Civil War, information on museums and the like.
So if you’re a publisher that has data, archeological information, for example, of people who are either doing tours or who want to go deeper because they’re actually doing some research into that spot, I think there’s a lot that can be done with the hands-free notion of a kind of layered involvement in content. But it’s going to take awhile.
KENNEALLY: Well, that’s nice because you’re walking around Washington and you want to know what’s there. But if you’re operating in the OR, it can also be applicable there, too, right?
STAINES: Yeah. There’s a lot of hospitals and doctor/patient situations where they’re using Glass. There’s a company I think called Wearable Intelligence that actually overwrite the operating system to disable some of the features that are causing privacy concerns, email sharing, that kind of thing. So there will be a role in those specialized contexts for wearables.
Industry, the petroleum industry, has taken a big interest in Glass, a lot of folks that have to go on location, go through security checklists, see schematics, you know, and the like. So it’s too early to say, you know, kind of how it’s going to play out. But I don’t think it has to be all things to all people. If it even shows promise, you know, in five areas, I think that will be a success.
KENNEALLY: Right. Andrea Powell, at CABI, your work takes you into developing parts of the world, in Africa and India and elsewhere. And I’m put in mind of another comment about the future from William Gibson, the science fiction writer, who said that the future is already here, it’s just not evenly distributed, and so while the future may be more distributed here in Washington, D.C., less so in Bali or Cameroon or someplace like that.
But you enter into a world where technology really matters immediately. People are using 2G phones, but the use of them is really transformative. Can you speak to that point?
POWELL: Yes, absolutely. It’s sometimes amusing to see the differences between the way we use mobile phone technology in the developed world to, you know, check in for flights, or check the weather, or the football scores or whatever, and contrast that to the way that telephones have been genuinely transformational in the developing world where you have whole populations who have never had any kind of interaction with valuable and authoritative data, suddenly finding the world in their hand. And the ability to deliver appropriate and very practical information, nuggets of information to those rural communities, to allow them to actually make a choice and make a difference to their lives, is very powerful.
As you say, we’ve been pioneering some mobile agri-advisory services in India and now expanding those into Africa, which have been hugely supported by the donor community, because they really genuinely see the ability to, like, cut out all the middle men and get right – information right into the hands of those who need it the most.
And it would be easy to be quite so patronizing if you like about the way that, you know, evolving, emerging markets are using technology. In fact, they can be very, very sophisticated in the way they use that technology, even though the technology per se could be quite low level. But the way they interact with each other and share that knowledge and work collaboratively to improve their collective experience is very humbling in many ways.
KENNEALLY: I hope you’re capturing that kind of data, the way they are working with it, because that could certainly inform future technologies.
David Martinsen, when it comes to speaking about the future, there are promises and there are problems. And I wonder whether you ever stop and ask yourself just how disruptive all these disruptors wind up being.
MARTINSEN: I do. But no, I think one of the things that I started thinking about today is we heard the two talks on analytics and metrics, is we have disruptors now in terms of these altmetrics. And the traditional metrics around citation and impact factor and even age index have been criticized. The problems with them have been pointed out.
And on the one hand, just an anecdote from a focus group that we had at one point. I had graduate students and they were talking about impact factor. And they all agreed it was a terrible measurement to assess journals. But then when we got into, like, search and discovery, they said, well, we’d like our search results sorted by impact factor.
So we questioned a little deeper: well, why do you say that? Well, because if they know the journal, they have certain journals they follow, they don’t need to worry about the impact factor for that. But nowadays, you do a search, you can get up with – come up with results that are obscure journals. And if you don’t know anything else about it, that impact factor has at least some meaning.
The other thing about altmetrics in particular is that we have a lot of different things that we can measure. But I don’t know for sure yet exactly what those measurements mean. So we have perhaps some common wisdom about first pitches or stealing bases. But when you actually have a greater degree of numbers, a greater degree of data to do more sophisticated analyses, then is when you can perhaps discover some of the nuances about which of those measurements are actually meaningful.
KENNEALLY: Howard Ratner, I see you nodding your head in agreement there. And it’s not just the growth in metrics, but also let’s admit there have been a number of standards that have been developed over time. And an important point there is that in the future we don’t necessarily want standards for their own sake. We need to have applicable uses for them.
RATNER: Right. So what we’ve seen over the last 25 years that I’ve been involved is that there are standards that have been created. They sit in some dusty coffer and never get used. And then there are the standards that never are done by a NISO, whatever, but become a standard, which is a de facto standard because they become operational. And all of these are important.
And what I like to say is, how they all meld together? How do they interoperate? And that’s really the important point. And that’s – you know, I know Todd. I don’t know if he’s here, but it’s one of the big things NISO is always looking at is how do these standards interoperate? And this is really important.
When I think of things, like – that I’ve been involved with like ORCHID, well how does ORCHID interoperate with DOIs? Well, it does. And that’s really good. Right? And then how do we – you know, then ORCHIDs and ISNIs? And how does that interoperate? And that’s really good. So it’s starting to work together.
And then you start to look at how all of these various different things mesh together. And it’s only then that you can actually create interesting applications. But coming back to the impact factor and thinking about the baseball analogies, yeah, impact factor is great. It’s one factor. Right? And you have to put it alongside other factors. And that’s where the real intelligence starts to happen. Right? Is, who makes the decision? What’s more important? Where’s the weighting on these decisions? You know, you can’t just throw away impact factor. It’s important.
But there are equally other important factors that are out there that have to be measured in there. And as we start to see things like ORCHID grow up and other things grow up, you’re going to start to see more interesting information happen.
The one thing I would encourage all of you to do is, it only becomes interesting when there’s enough data, you know, in that machine. If there’s only a few entries in the machine, it’s not interesting. If you have millions of records or millions of games, it actually is interesting. So I encourage you all to participate so we can make this stuff interesting.
KENNEALLY: Well, with that then, I want to thank our panel, Heather Staines, David Martinsen, Andrea Powell, and Howard Ratner. Thank you.